-- Logs begin at Mon 2022-08-29 20:49:13 UTC, end at Mon 2022-08-29 21:52:29 UTC. -- Aug 29 21:19:42.760201 dv-cinder-854266-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 21:19:42.768223 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 21:19:42 2022] *** Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: nodename: dv-cinder-854266-3-iscsi Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: machine: x86_64 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: clock source: unix Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: pcre jit disabled Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: detected number of CPU cores: 4 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: current working directory: / Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: detected binary path: /usr/bin/uwsgi-core Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: your processes number limit is 47680 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: your memory page size is 4096 bytes Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: detected max file descriptor number: 2048 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: lock engine: pthread robust mutexes Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: thunder lock: enabled Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 21:19:42.773498 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 21:19:42.813758 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: Python main interpreter initialized at 0x55c2e86349c0 Aug 29 21:19:42.813758 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: python threads support enabled Aug 29 21:19:42.813758 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: your server socket listen backlog is limited to 100 connections Aug 29 21:19:42.813758 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: your mercy for graceful operations on workers is 90 seconds Aug 29 21:19:42.814172 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 21:19:42.814250 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: *** Operational MODE: preforking *** Aug 29 21:19:42.814481 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: *** uWSGI is running in multiple interpreter mode *** Aug 29 21:19:42.814481 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: spawned uWSGI master process (pid: 81530) Aug 29 21:19:42.814711 dv-cinder-854266-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 21:19:42.820611 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: spawned uWSGI worker 1 (pid: 81536, cores: 1) Aug 29 21:19:42.820611 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: spawned uWSGI worker 2 (pid: 81537, cores: 1) Aug 29 21:19:42.820611 dv-cinder-854266-3-iscsi devstack@keystone.service[81530]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 21:19:47.186424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:19:47.186424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Aug 29 21:19:47.187197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:19:47.187197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: from cryptography.utils import int_from_bytes Aug 29 21:19:47.187310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:19:47.187310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Aug 29 21:19:47.187733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:19:47.187733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: from cryptography.utils import int_from_bytes Aug 29 21:19:47.237792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 21:19:47.237998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 21:19:47.238090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 21:19:47.238228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 21:19:47.238346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] command line args: [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 21:19:47.238462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] command line args: [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 21:19:47.238554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 21:19:47.238631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 21:19:47.238693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ================================================================================ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 21:19:47.238775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ================================================================================ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 21:19:47.238894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.238984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_dir = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_dir = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] config_source = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] config_source = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] debug = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] debug = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.239834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.240965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_config_append = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_config_append = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_dir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_dir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.241952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.242848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.243589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.244970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] publish_errors = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.245967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] publish_errors = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.246979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] standard_threads = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] standard_threads = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.247976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_journal = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_journal = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_json = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_json = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_stderr = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_stderr = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] use_syslog = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] use_syslog = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:19:47.248954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.249957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.250944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.251933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.252014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.252706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.252807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.252962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.253970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.254974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.255952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.256950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.257996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.258958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.259915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.260859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.261976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.262928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.263971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.264924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.265912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.266878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.267778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.268985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.269968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.270924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.271939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.272965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.273898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.274854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.275994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.external = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.external = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.276857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.password = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] auth.token = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.password = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] auth.token = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.277969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.278785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.279966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.280976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.281965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.282991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.283172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.283376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.283575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.283850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.284938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.285891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.286797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.287841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.288904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.289870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.290954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.291922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.292913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.293915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.294845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.295962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.296934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.297990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.298905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.299474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.299638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.299757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.299866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.299933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.300942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.301972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.302845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.303990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.304959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.305898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.306988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.307857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.308929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.309963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.310972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.311939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.312953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.313925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.314104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.314175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.314354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.314446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.315997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.316157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.316265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.316529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.316751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.317213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.317297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.317669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.317757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.318959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.319988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.320826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.321896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.322953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.323956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.324958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.325944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.326963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.327942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.328951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.329960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.330034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.330822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.330915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.331978 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.332069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.332220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.332302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.332397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81537) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 21:19:47.332478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.332744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.333001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.333243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.333493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.333731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.333978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.334223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:19:47.334393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81536) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 21:19:47.362499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.362734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.362961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.363389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.363624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.363852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.371425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.372348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.400907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.401116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.404520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.404781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.456105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.457835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.505074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.505310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.507660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.507901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.664133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.664382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.664618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.664768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.664859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.665863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.692015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.694914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.695003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.695003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.695003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.704183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.704355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.718554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.718769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.719214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.719346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.719470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.719727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.719893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.720160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.720245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.720323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.720509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.720730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.721325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81537) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.721395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81536) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:19:47.738355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.738599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:19:47.738906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.739056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.739205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:19:47.739346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:19:47.739483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.739602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.739758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.739900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:19:47.740041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.740174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.740307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.740460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.740637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.740778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.749902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.750265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.750459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.750651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.750784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.751060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.751157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.751443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.758376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.758536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.761323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.761463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.761614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.761768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.761908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.762841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.763931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.764027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.764203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.764289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.787645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.788130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.789172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.789469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.789667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.789957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.794179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.794467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.794594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.794749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.794839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.795049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.801715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.802006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.802211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.802508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.806111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.806399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.806529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.806688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.806776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.807883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.808141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.809778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.810071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.810359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.825741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.825929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.826031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.826154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.829969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.830104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.833075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.833252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.833364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.833552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.837669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.837836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.840768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.840997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.843832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.844810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.845011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.845093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.845352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.845434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.845636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.862807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.863184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.863522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.863663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.863901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.863987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.864248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.864516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.871248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.874688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.874812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.874908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.874908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.875043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.876874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.877183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.877312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.877606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.878019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.878312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.878609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.880376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.880698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.881145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.881492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.881793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.889868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.890162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.890553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.890843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.891212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.891414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.891714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.894676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.894983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.895271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.895556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.895842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.896121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.896409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.907326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.911096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.912715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.914020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.915486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.915774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.917125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.917410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.917710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.918760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.920216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.921698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.921995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.922294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.924653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.926153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.929194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.929442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.929535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.929767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.930046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.930911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.933831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.934133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.934422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.934711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.937904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.938206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.938493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.942726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.943024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.943317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.945623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81536) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.945909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.946233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.946473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.946745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.947019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.947295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.947579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.947860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.948141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.948413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.948717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.948985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.949270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81536) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.950288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81537) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:19:47.950582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.950868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.951148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.951423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.951700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.951977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.952256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.952534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.952843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.953117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.953380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.953899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.953899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81537) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:19:47.984287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c2e86349c0 pid: 81536 (default app) Aug 29 21:19:47.988263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ae2e-8414-4aa7-9bef-a2b31f9ecbce None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:47.988553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ae2e-8414-4aa7-9bef-a2b31f9ecbce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:47.988854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ae2e-8414-4aa7-9bef-a2b31f9ecbce None None] PATH_INFO: `/v3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:47.989182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c2e86349c0 pid: 81537 (default app) Aug 29 21:19:47.993119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Aug 29 21:19:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:19:54.854051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:54.854631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:54.854631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:54.858935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:19:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:19:54.865238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:54.866159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:54.866159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:54.966815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: INFO dbcounter [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] Registered counter for database keystone Aug 29 21:19:54.987647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG oslo_db.sqlalchemy.engines [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81536) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 21:19:54.997288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writer thread running {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 21:19:55.220983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81536) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 21:19:55.221289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG passlib.handlers.bcrypt [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81536) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 21:19:55.282437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:55.285298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:55.433011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 29 21:19:55.951726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:56.022761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: INFO dbcounter [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] Registered counter for database keystone Aug 29 21:19:56.044699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG oslo_db.sqlalchemy.engines [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81537) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 21:19:56.051452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writer thread running {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 21:19:56.171069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20df87a-fac6-4523-9a76-b01a15ae2fca None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:56.176057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:56.176750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:56.177052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:56.177320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:56.185762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:56.237039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING keystone.common.rbac_enforcer.enforcer [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 21:19:56.237784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4091-13ac-4bb7-a440-161c5d64e73e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:56.247591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 29 21:19:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:19:56.253689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:56.268731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44a627-3071-4271-b848-a0e4aeb350d6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:56.273293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:56.274410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:56.274694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:56.274966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] PATH_INFO: `/v3/projects/918734ce2c8c4362a4d1364fce951bb2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:56.276627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] RBAC: Authorizing `identity:get_project(project_id=918734ce2c8c4362a4d1364fce951bb2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:56.329540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING keystone.common.rbac_enforcer.enforcer [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 21:19:56.330475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dc567-9378-4a6d-ad2d-7f7254732420 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:56.335277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Aug 29 21:19:56 2022] GET /identity/v3/projects/918734ce2c8c4362a4d1364fce951bb2 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:19:58.253605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71c5f0-5790-4d19-9f9d-ceaa6ce0719d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:58.254156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71c5f0-5790-4d19-9f9d-ceaa6ce0719d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:58.254156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71c5f0-5790-4d19-9f9d-ceaa6ce0719d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:58.258178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:19:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:58.263873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:58.264281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:58.264429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:58.285762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:58.287303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:58.295250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 21:19:58.560970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71c5f0-5790-4d19-9f9d-ceaa6ce0719d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:58.573947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71c5f0-5790-4d19-9f9d-ceaa6ce0719d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:58.578235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:58.578835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:58.579113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:58.579383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:58.587800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:58.589619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2960ece-259c-478f-a293-4102ffc9bedc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:58.599484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 29 21:19:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:19:58.605390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:58.615641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2bd2c5-af3b-4b0b-8312-0b9862171e55 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:58.620277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:58.620953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:58.621245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:58.621523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:58.622843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:58.624726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8eb3c-7d77-47cc-83a9-c921a7288962 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:58.626305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Aug 29 21:19:58 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:00.636798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7978e-23fa-4556-b116-66b891d120ef None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:00.637403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7978e-23fa-4556-b116-66b891d120ef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:00.637403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7978e-23fa-4556-b116-66b891d120ef None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:00.638698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:00.644349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:00.645015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:00.645406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:00.669336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:00.670851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:00.702331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df3d-8540-4a38-9a4f-a5e7c92766ba None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:00.702592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df3d-8540-4a38-9a4f-a5e7c92766ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:00.702871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577df3d-8540-4a38-9a4f-a5e7c92766ba None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:00.704215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:00.710115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:00.710401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:00.710680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:00.756911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81537) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 21:20:00.757312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG passlib.handlers.bcrypt [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81537) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 21:20:00.779676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:00.809137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:00.811350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:00.938891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:01.053149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.078271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.165927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68a99-294d-4cf5-91b8-39d03be9ebe6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.201939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.202504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.202786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.203068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.215849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.216107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:01.224880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:01.225869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:01.226721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:01.231943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.323232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83749-0e34-404f-9fd0-eafbd0463696 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.323901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.325624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.344375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02bae9-44e8-40f1-9571-c02b90bd2c3c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.364755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.365761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.366039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.366306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.380996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.381605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.381893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.382166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.382851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:01.383159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.384570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.397523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:01.403051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.440947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.442633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.516016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa41974-e67d-4cb8-af94-f76b78df15de None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.546146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:20:01.552251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.552388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.553032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.553314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.553581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.554738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.556164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.556252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.557824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:01.563592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.578996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.677855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a23ef-08bd-4269-8c7e-a82e16fd1b13 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.680182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f4455-1350-4988-8fe3-5d9bb9745997 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.719267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.720130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.720235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.720412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.720762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.720865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.721091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.721368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.722516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.723996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.725652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:01.731623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.741133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.742637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:20:01.763167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:01.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:20:01.765045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:20:01.765045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:20:01.765045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:01.765045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:01.770188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:01.816345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.830549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:01.838695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750efb-2cc3-44d9-938f-66dc9379fda9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.878737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.879360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.879643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.879917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.880700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.domain.id=default, user.id=3cc351f3cc8d45098e84aaabaeda36e6) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:01.881006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.882360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f37c39-9155-4e15-8ec2-56444fc4bea3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:01.882485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059c4a8-16ad-43d9-b112-40bb6472b1dd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.889990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/role_assignments?user.id=3cc351f3cc8d45098e84aaabaeda36e6&scope.domain.id=default&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:01.924782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:01.925423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:01.925699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:01.925973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:01.926670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:01.927019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:01.928473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7750e-7674-4dc0-806a-e7e97a654ed6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:01.949811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:04.454390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed1624-d0e5-4dde-986c-7f551f8be855 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.455236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed1624-d0e5-4dde-986c-7f551f8be855 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.455236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed1624-d0e5-4dde-986c-7f551f8be855 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.456357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:04.461852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d66-592e-4808-89e2-69b595318b0d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.462493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d66-592e-4808-89e2-69b595318b0d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.462493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d66-592e-4808-89e2-69b595318b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.491175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-88596d66-592e-4808-89e2-69b595318b0d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:04.492383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-88596d66-592e-4808-89e2-69b595318b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:04.589746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:04.617424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f3bc-85af-4977-a135-5f278c27a06e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.617828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f3bc-85af-4977-a135-5f278c27a06e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.617966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f3bc-85af-4977-a135-5f278c27a06e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.619322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:04.628636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829742e9-35d9-4192-924a-925a36daa3a0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.628878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829742e9-35d9-4192-924a-925a36daa3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.629175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829742e9-35d9-4192-924a-925a36daa3a0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.630494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 14/25] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:04.630841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b634-e7ef-44df-899e-41ec8ba7f4a3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.631129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b634-e7ef-44df-899e-41ec8ba7f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.631400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b634-e7ef-44df-899e-41ec8ba7f4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.638462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.638732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.639010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.668134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:04.669353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:04.676421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-02f0b634-e7ef-44df-899e-41ec8ba7f4a3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:04.677647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02f0b634-e7ef-44df-899e-41ec8ba7f4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:04.796310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:04.858028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 15/26] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:04.866856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 12/27] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:04.941554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.039381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:05.075459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2e1c7-15fe-4565-b723-6d57f0b62a34 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.106997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc999-ff14-4c92-a6c7-166bbec38778 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.107316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc999-ff14-4c92-a6c7-166bbec38778 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.107545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc999-ff14-4c92-a6c7-166bbec38778 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.108908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 13/28] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:05.117986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.118566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.119056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.134105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:05.134992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.135275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.135549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.137300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:05.138794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:05.161642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:05.163200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:05.321977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 14/29] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:05.351131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 16/30] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 29 21:20:04 2022] POST /identity/v3/roles => generated 225 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:20:05.421290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.481320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.570522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fa9a8-b127-4d29-9dd6-c6bf3f3e1168 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.578684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:05.630781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:05.631349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.631629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.631902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.658290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:05.663578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:05.664526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:05.665338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 15/31] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:05.676636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.707069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c162-4347-417c-ab6a-90bcb9681805 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.757444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:05.758036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.758322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.758602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.763965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:05.777125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:05.779114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:20:05.799146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:05.800356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:20:05.801395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:20:05.801395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:20:05.801395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:05.805143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 17/32] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:05.806190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.809975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638c28-08d6-4b6a-8d61-937fffca354b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.850777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:05.852568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.853693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.854190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.854857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:05.855157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:05.856555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:05.869971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 16/33] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:05.870984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.903733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:05.952132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9da651-97e0-4c90-a447-213738648bde None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.979661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f01bc-0650-471a-bd5d-721a64013af5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:05.991267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:05.992165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.992620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.993071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.994001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:05.995663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.015182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 18/34] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 29 21:20:05 2022] POST /identity/v3/roles => generated 219 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:06.016335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.019382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.020716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.022185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.022686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.023517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:06.023986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.025609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.042429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 17/35] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:06.050962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.137304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:06.238173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce0eac-780b-4972-9037-7e1fcfd52529 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:06.240280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:06.257652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a7c4-0eed-43f9-818f-811f1d0a8839 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:06.279332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.280235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.280753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.281195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.282943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.284779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f229e-110b-420f-a174-a4c0768aa731 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.286693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 19/36] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:06 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:06.288198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902e8b2-cc4c-421f-bbfd-e9824066b898 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.288695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902e8b2-cc4c-421f-bbfd-e9824066b898 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.289121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902e8b2-cc4c-421f-bbfd-e9824066b898 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.290627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 20/37] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:06.294463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902e8b2-cc4c-421f-bbfd-e9824066b898 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.301660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.302312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.302598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.302878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.303833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.305425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a8745-5f9b-445a-8971-9cec35445536 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.420342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902e8b2-cc4c-421f-bbfd-e9824066b898 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:06.436518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:06.450501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 18/38] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:20:06 2022] POST /identity/v3/projects => generated 301 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:06.452099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440291-d238-4c5e-987c-747420ad048c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.452384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440291-d238-4c5e-987c-747420ad048c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.452702 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24440291-d238-4c5e-987c-747420ad048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.469730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.470398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.470952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.471220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.472458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.473989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-24440291-d238-4c5e-987c-747420ad048c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:06.474182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e1a16-14f3-45cc-9813-d975e2983979 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.475589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-24440291-d238-4c5e-987c-747420ad048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:06.481090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 21/39] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:06.481090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745487df-8376-4c45-a88f-a56512d74452 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.481090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745487df-8376-4c45-a88f-a56512d74452 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.481090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745487df-8376-4c45-a88f-a56512d74452 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.481090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 22/40] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:06.482016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745487df-8376-4c45-a88f-a56512d74452 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.565096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 19/41] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:06.566733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.567024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.567301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.589216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:06.590619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:06.623694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745487df-8376-4c45-a88f-a56512d74452 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:06.671216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:06.671523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.672532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.672532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.672803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] PATH_INFO: `/v3/domains/default/users/3cc351f3cc8d45098e84aaabaeda36e6/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.675924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3cc351f3cc8d45098e84aaabaeda36e6, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.677665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.678488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 20/42] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:06.699256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 23/43] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Aug 29 21:20:06 2022] PUT /identity/v3/domains/default/users/3cc351f3cc8d45098e84aaabaeda36e6/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:20:06.856206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.961472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:06.972671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789127ab-3ef4-4222-892f-b60a6733d443 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:07.013521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:07.017036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:07.017712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.017994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.018273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.019409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:07.021107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:07.023045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 21/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:07.028573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:07.093881 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ac7a7-2e28-499d-bf90-719fe1d82531 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:07.106747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:07.137868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:07.142967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.143252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.143530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.144256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:07.145737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff03fa-d9eb-453d-800c-c9264b243134 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:07.159995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df51dd-ee9d-412b-a22c-b717631ac21c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:07.165329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 24/45] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 29 21:20:06 2022] POST /identity/v3/roles => generated 223 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:20:07.215074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:07.215727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.216018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.216298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.217058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:07.218668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44010141-b64c-40f8-9be4-04e23bc81aa7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:07.252325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:20:07 2022] POST /identity/v3/projects => generated 312 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:20:08.778730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561122ff-cc10-46d3-bfa3-e4490cc85a8c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:08.779249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561122ff-cc10-46d3-bfa3-e4490cc85a8c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:08.779334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561122ff-cc10-46d3-bfa3-e4490cc85a8c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:08.780693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:08.787594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef8a-d920-49ad-b704-03161cf58f0e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:08.788345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef8a-d920-49ad-b704-03161cf58f0e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:08.788706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21ef8a-d920-49ad-b704-03161cf58f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:08.807910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa21ef8a-d920-49ad-b704-03161cf58f0e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:08.809144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa21ef8a-d920-49ad-b704-03161cf58f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:08.880629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:09.147870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561122ff-cc10-46d3-bfa3-e4490cc85a8c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.216652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c861b6-1dee-46a8-b27f-c9e39214f0f4 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.216933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c861b6-1dee-46a8-b27f-c9e39214f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.217372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c861b6-1dee-46a8-b27f-c9e39214f0f4 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.218730 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 24/49] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:09.223976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.224774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.225069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.243532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:09.244776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:09.265624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561122ff-cc10-46d3-bfa3-e4490cc85a8c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.268213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.307079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.307820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.308474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.308771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.309265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.325979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:09.331059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:09.331982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:09.332793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 26/50] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:09.333458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 25/51] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:09.338672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.558613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968157e2-28bb-4cd4-a431-d2649873ae5a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.594779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.595880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.596161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.596434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.597198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:09.597447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:09.598843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:09.607297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:09.613622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.614230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.642389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.730789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b3c1-2823-4a7c-95c3-88d29b2fb9eb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.731176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a708cca-ecf5-45a3-beb4-57ef20c5acf5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.769201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.769855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.770137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.770410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.771632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:09.773271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:09.775242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 28/53] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:09.781114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.885118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.885773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.886058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.886341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.887736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:09.889233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:09.891014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 26/54] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:09.897297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.897998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.914585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415863b-cb81-47f5-8506-c38b2681b0e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.956643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.957275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.957563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.957842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.958277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.958512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:09.960126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:09.993679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 29/55] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 29 21:20:09 2022] POST /identity/v3/projects => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:20:10.023778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cab7e-565d-4117-ba65-96f05c947c75 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:10.081690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:10.082738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:10.082885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:10.083328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:10.085594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:10.087635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5614a-c930-438c-a3fe-c80410a70518 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:10.089970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:10.097566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:10.230796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ba98-3339-4bac-993d-b3e679108463 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:10.274800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:10.275425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:10.275709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:10.275987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:10.276764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.domain.id=default, user.id=3cc351f3cc8d45098e84aaabaeda36e6) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:10.277076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:10.278533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7919-6453-416d-a82a-6ed2e197ca8f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:10.284917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 30/57] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 29 21:20:10 2022] GET /identity/v3/role_assignments?user.id=3cc351f3cc8d45098e84aaabaeda36e6&scope.domain.id=default&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 530 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:11.983467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1ce3-26af-40f3-b157-a0707b33ded9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:11.983982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1ce3-26af-40f3-b157-a0707b33ded9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:11.984368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1ce3-26af-40f3-b157-a0707b33ded9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:11.986566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:11.992062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:11.992536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:11.992642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:12.011484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:12.013070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:12.015092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.082982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:12.350804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1ce3-26af-40f3-b157-a0707b33ded9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:12.451900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1ce3-26af-40f3-b157-a0707b33ded9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:12.467959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.487106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:12.487739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:12.488020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:12.488294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:12.489458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:12.490894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c5578-d37d-4537-b054-c3f0b1323ae1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:12.492470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:12.498097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:12.596264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280a4b4-4dda-4c3b-81fa-97c9e5b24faf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:12.630438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:12.630971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:12.631253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:12.631557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:12.632269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:12.633830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a898d-311f-487f-8a67-01a598dbce0d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:12.648821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.662690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:12 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:15.608370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac058-5a4e-4434-9f60-342c7fc161be None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:15.609448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac058-5a4e-4434-9f60-342c7fc161be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:15.609932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac058-5a4e-4434-9f60-342c7fc161be None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:15.612052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:15.617666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd194-7f9d-4d14-bf13-bac45737cc5e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:15.618539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd194-7f9d-4d14-bf13-bac45737cc5e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:15.619012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dd194-7f9d-4d14-bf13-bac45737cc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:15.645566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-112dd194-7f9d-4d14-bf13-bac45737cc5e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:15.646760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-112dd194-7f9d-4d14-bf13-bac45737cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:15.741934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:16.057313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0e99-859d-4f83-beff-25c30522242f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.058017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0e99-859d-4f83-beff-25c30522242f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.058495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f0e99-859d-4f83-beff-25c30522242f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.060107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 31/64] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:16.065365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d692e-ac95-4a81-9623-6d0d2ac71582 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.065909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d692e-ac95-4a81-9623-6d0d2ac71582 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.066399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d692e-ac95-4a81-9623-6d0d2ac71582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.097699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-907d692e-ac95-4a81-9623-6d0d2ac71582 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:16.099240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-907d692e-ac95-4a81-9623-6d0d2ac71582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:16.145398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f0e99-859d-4f83-beff-25c30522242f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:16.193070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:16.277533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f0e99-859d-4f83-beff-25c30522242f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:16.315622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:16.316669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.317169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.317640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.329041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:16.334245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:16.335197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:16.336000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:16.338373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 32/66] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:16.343083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b01cf-1882-44af-ad0e-1619bd406ff1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.343381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b01cf-1882-44af-ad0e-1619bd406ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.343661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b01cf-1882-44af-ad0e-1619bd406ff1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.345838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 35/67] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:16.352919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c757a-1adc-4aee-a6ba-f9870c5b0a57 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.353209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c757a-1adc-4aee-a6ba-f9870c5b0a57 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.353487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c757a-1adc-4aee-a6ba-f9870c5b0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.357453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:16.381494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6f3c757a-1adc-4aee-a6ba-f9870c5b0a57 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:16.382701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f3c757a-1adc-4aee-a6ba-f9870c5b0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:16.453523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:16.466920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:16.491537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 36/68] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:16.493045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6f36-7e13-4f96-bdbc-35bfbe3dbc69 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.493403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6f36-7e13-4f96-bdbc-35bfbe3dbc69 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.493745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb6f36-7e13-4f96-bdbc-35bfbe3dbc69 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.495055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 37/69] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:16.516395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b6992-595c-440e-a977-ca3d74399ba7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.516709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b6992-595c-440e-a977-ca3d74399ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.517047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b6992-595c-440e-a977-ca3d74399ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.537758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-076b6992-595c-440e-a977-ca3d74399ba7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:16.538946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-076b6992-595c-440e-a977-ca3d74399ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:16.558063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca2f2-7ae2-47f9-a4b6-6b9e07571168 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:16.608929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:16.609797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.610228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.610534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.611798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:16.612119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:16.614926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:16.632910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 33/70] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:16.634690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:16.667699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 38/71] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:16.667699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e42fe-4450-4ce2-b0e1-d10cace3c405 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.667699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e42fe-4450-4ce2-b0e1-d10cace3c405 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.667699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e42fe-4450-4ce2-b0e1-d10cace3c405 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.675268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 39/72] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:16.677056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e42fe-4450-4ce2-b0e1-d10cace3c405 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:16.900485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ffdd0-9131-4404-9e02-c3158d246eaf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:16.912122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:16.913281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e42fe-4450-4ce2-b0e1-d10cace3c405 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:16.940152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:16.959004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:16.959915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.960208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.960489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.968721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:16.969386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.969680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.969964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.973343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c58cff67-0d4f-4529-9831-53bd350f13f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:16.979683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:16.981156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 34/73] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:16.982520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:16.983711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:16.983711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:16.995260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:16.997475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.000721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 40/74] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:16 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.004815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.005803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:17.007260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:17.107068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 35/75] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:17.107946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.146459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69201d-d982-452d-9c1d-cb2c35bd3a23 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.191608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.192698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.193170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.193642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.210334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:17.216615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:17.217601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:17.218430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 41/76] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:17.219230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.231347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.234030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606ad0-511e-4212-8657-d13d5b90c1f4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.259518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.278583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.279309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.279600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.279887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.280567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:17.280962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.282302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.293206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 36/77] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.294066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.352266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8ddde-a344-4d75-bbf2-653ec310181d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.389267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.390051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.390444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.390808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.409180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2fb50-aa6c-4697-bb6b-2e3166f1e21f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.449499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.450561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.450949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.451231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.452711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.454175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.456300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.457102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.469761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.471841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.490427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.504078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.574980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf446ab-e586-4609-905c-5ec1f62fd104 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.580231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 42/79] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:17.581206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.615490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.616137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.616391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.616698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.617434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:17.617737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.619129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.628147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 38/80] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.628987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.696133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2346dd3-78ab-443e-ae27-0af28a608357 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.726631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.731526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.737709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.738369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.738976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.738976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.740094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.741603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.741696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60c14-dadf-4d81-861f-13715825f1e9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.743387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 43/81] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.744095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.781347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.782085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.782419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.784333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.794010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:17.799933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:17.801056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 39/82] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:17.803500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.864091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc0f10-2217-44cd-8645-30dd14cfbdf9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.898857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.899474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.899758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.900036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.900814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.902290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.912125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735ae35-534b-4146-8c66-775d93053515 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:17.926415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 44/83] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:17 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:17.927488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.950268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:17.950848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.951135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.951415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.952705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:17.954205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:17.956242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 40/84] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:17.957149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:17.987887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.018843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.067899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915231c8-4843-4a4f-9ad3-9da18fdc71a1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.092197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c372-7dfb-4c44-95a6-37e6ad135224 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.112056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.112752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.113035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.113315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.114485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.115937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.117564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 45/85] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:18.118473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.135190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.135819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.136108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.136385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.224720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.226310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.241496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad335386-93fb-4a5a-bcf6-9b4bc4aaefd8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.247179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.273583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.288927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.289551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.289901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.290182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.290862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:18.291166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.292558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.301742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 46/86] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:18.302588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.368254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 41/87] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:18.369773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.417222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6c09-51d9-4357-83b5-e2aa8cfcb93a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.457433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.458061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.458434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.458723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.459719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:18.459892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.461354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.469300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 47/88] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:18.476714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.508890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9a10-0370-457b-9dc5-0b09865b6757 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.530772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.557483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.558437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.558922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.559387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.593370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.658713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6533bb-3f86-4ae3-a87c-5a6d5d7fb50c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.704649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.707077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.707542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.707983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.710646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.713305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.716961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 48/89] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:18.722383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.752745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.752745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.903042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 42/90] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:18.912961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.956942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.010121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe631a-4eda-40a8-b485-9b2073f9db81 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.045572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.100231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d273299-6761-4a1f-8b56-bb721d9d377e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.110897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.111777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.112075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.112359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.117320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.118820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.120530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 49/91] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:19.121336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:19.138713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.139788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.140084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.140379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.219665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.221244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.242558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.334537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 43/92] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:19.335647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:19.369536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68457783-637d-4c1b-85e3-1340131d6fd3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.410087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.410745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.411013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.411291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] PATH_INFO: `/v3/projects/d3e13b1664054eb1ad670422685b6ba8` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.412443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] RBAC: Authorizing `identity:get_project(project_id=d3e13b1664054eb1ad670422685b6ba8)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.420619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.420619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 50/93] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:19 2022] GET /identity/v3/projects/d3e13b1664054eb1ad670422685b6ba8 => generated 312 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:19.420619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:19.485042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.538976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4e7f8-84f6-4990-8afe-bfa960265d66 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.588392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.589039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.589325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.589601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.590841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.592726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.594994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 51/94] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:19 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:19.596497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:19.622837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee38c8b-7898-493e-a752-a835e6c39874 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.676518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.735267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab812-4b80-4ff7-82a5-9242ec35393c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:19.758771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.760111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.760622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.760907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.761648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=3cc351f3cc8d45098e84aaabaeda36e6) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:19.761971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.764845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7708f9-d55d-4cb4-b1fd-e7effcfced21 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.775053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:19.783689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:19.784546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:19.784987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:19.785344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:19.786159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=d3e13b1664054eb1ad670422685b6ba8, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:19.786550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:19.788098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:19.795202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 52/95] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:19 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=d3e13b1664054eb1ad670422685b6ba8&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:19.796204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:19.799660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 44/96] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:19 2022] GET /identity/v3/role_assignments?user.id=3cc351f3cc8d45098e84aaabaeda36e6&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:19.956509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca227be6-e12c-4d8b-87c1-7ff5fd297b64 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:20.027059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:20.028148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.028638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.029142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.030381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:20.030910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:20.033764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c731-12da-406f-a1fa-20b1b94699d8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:20.044533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 53/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:19 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=2be7d12a9a644f299290473830d5c537 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:20.348973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e215a3-5680-4949-9256-d2a05ccf5c37 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.349385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e215a3-5680-4949-9256-d2a05ccf5c37 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.349561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e215a3-5680-4949-9256-d2a05ccf5c37 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.351046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 45/98] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:20.367433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120a799-07d5-4b02-bc81-97031fd6c098 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.367955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120a799-07d5-4b02-bc81-97031fd6c098 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.368461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120a799-07d5-4b02-bc81-97031fd6c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.411795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3120a799-07d5-4b02-bc81-97031fd6c098 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:20.413777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3120a799-07d5-4b02-bc81-97031fd6c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:20.520989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:20.566806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 54/99] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:20.946339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c726d6-14b4-4bf8-be67-be9edc760166 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.947858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c726d6-14b4-4bf8-be67-be9edc760166 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.948680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c726d6-14b4-4bf8-be67-be9edc760166 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.951334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 46/100] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:20.957205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.957696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.957767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.986533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:20.988182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:21.075544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 55/101] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:21.117571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c726d6-14b4-4bf8-be67-be9edc760166 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:21.269875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c726d6-14b4-4bf8-be67-be9edc760166 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:21.327267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:21.328866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.330201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.330968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.353911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:21.364519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:21.366348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:21.367844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:21.372272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 47/102] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:21.379590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:21.515582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:21.646162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:21.656263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8272b62-1336-4586-a1c4-1931e77824f6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:21.685765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:21.746397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:21.747043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.747320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.747606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.748281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:21.748601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:21.752616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8560de9-40c1-4ccd-b9ba-7e8e1a41f17c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:21.768805 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 56/103] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:21.774364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032eac-94f0-47bd-821a-d58ee922696e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.774731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032eac-94f0-47bd-821a-d58ee922696e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.774997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44032eac-94f0-47bd-821a-d58ee922696e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.776307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 57/104] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:21.777073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44032eac-94f0-47bd-821a-d58ee922696e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:21.792839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6226c-2534-49ca-8052-94609aec2c99 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:21.863339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:21.863991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.864313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.864658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.866082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:21.867674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f1a7-6601-4d48-beb9-54c17cd643d9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:21.874039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 48/105] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:21.874039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.874039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.874039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.912966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:21.914090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:21.971999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44032eac-94f0-47bd-821a-d58ee922696e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:22.068210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:22.068889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.069179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.069463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] PATH_INFO: `/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.090125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 49/106] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:22.091173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:22.116617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] RBAC: Authorizing `identity:get_user(user_id=fe4e43e5efba467bab6c9cd3d18ca28f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:22.116617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:22.117106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 58/107] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:21 2022] GET /identity/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f => generated 289 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:22.125441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:22.169199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:22.189246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:22.273296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ddea0-f976-402d-be04-29af6ac2f126 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:22.306662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd68ae8-c0ee-4775-89a4-4afcdbbb8163 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:22.322551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:22.323195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.323484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.323797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.324476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:22.325988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483265f6-f775-4875-9d8b-eee58b189a5f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:22.370448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:22.371649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.372167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.372679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.393620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 50/108] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:20:22 2022] POST /identity/v3/projects => generated 302 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:22.455107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967ebc-5c43-471d-83af-90f863594d40 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.455943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967ebc-5c43-471d-83af-90f863594d40 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.456237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69967ebc-5c43-471d-83af-90f863594d40 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.458251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 51/109] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:22.464622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.465083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.465510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.503748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:22.505355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:22.515227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:22.517419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:22.573443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:22.633786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 52/110] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:22.634907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:22.709193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:22.739118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 59/111] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:20:22.759267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13c798-f75d-4c16-a5d9-0acd4a6752c8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:22.762619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:22.800966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:22.801633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:22.801918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:22.802195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:22.823611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:22.831576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:22.832486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:22.833305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 53/112] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:22.841558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:22.937896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cdf36-4e55-4753-a1cb-02aa5bdad799 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.012290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.024072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.025039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.025469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.025928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.029415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:23.031142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:23.033637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 60/113] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:22 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:23.040456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.087441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.146876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c801fb-98ba-4297-af3d-99f11d997b94 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.195049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f655-8ce6-41ee-809e-b41de359594e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.230786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.231413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.231696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.231976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.232672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:23.232980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:23.234383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:23.244164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 54/114] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:23.248835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.251109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.251789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.252053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.252380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.253534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=fe4e43e5efba467bab6c9cd3d18ca28f) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:23.253844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:23.256429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403af9e-51ff-4390-be8f-64bfe1a47749 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:23.263874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 61/115] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/role_assignments?user.id=fe4e43e5efba467bab6c9cd3d18ca28f&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:23.269372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0249-f1ce-4bdf-92a0-c6d12f1e0dd0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.269970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0249-f1ce-4bdf-92a0-c6d12f1e0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.270476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab0249-f1ce-4bdf-92a0-c6d12f1e0dd0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.272225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 62/116] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:23.279514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab0249-f1ce-4bdf-92a0-c6d12f1e0dd0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.428117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.431468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b64a-a998-42a5-abdd-4d40aae8b62b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.473697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab0249-f1ce-4bdf-92a0-c6d12f1e0dd0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.484735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.485787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.486274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.486879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.499001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e2f05d8e-9386-400d-9f8e-c1aa8d0643ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:23.505195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:23.506105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:23.506906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:23.509432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 55/117] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:23.510964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbf8-d028-472c-9957-0454e35070dc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.511472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbf8-d028-472c-9957-0454e35070dc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.512114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eccbf8-d028-472c-9957-0454e35070dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.513290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.513989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.514211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.514635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.515698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:23.517978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9808d431-2c15-4a87-b8d5-6a8f2a6da5a7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:23.519734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 63/118] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:23.521043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274a798-fcbe-4b68-8ec4-9a59c84470da None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.521327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274a798-fcbe-4b68-8ec4-9a59c84470da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.521670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274a798-fcbe-4b68-8ec4-9a59c84470da None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.522833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.522940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 64/119] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:23.523635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274a798-fcbe-4b68-8ec4-9a59c84470da None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.538784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a6eccbf8-d028-472c-9957-0454e35070dc None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:23.540365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6eccbf8-d028-472c-9957-0454e35070dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:23.634116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 56/120] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:23.635676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.636474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.637192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.659601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:23.660894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:23.667696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274a798-fcbe-4b68-8ec4-9a59c84470da None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:23.711477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.723231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:23.723849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.724336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.724629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.725310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:23.725611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:23.727003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:23.750709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 65/121] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:23.752286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.837159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 57/122] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:23.838757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:23.940962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.985424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e6724-1e75-4351-a0f7-817aee19a54f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.025986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.027126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.027653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.028238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.030027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.031695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.034508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 58/123] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:24.035463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.065219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2636-2469-4a8b-b9e9-b3356f48c664 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.154882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402a5ca-3345-43ed-be0b-732d0abeee40 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.169532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.170033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.170319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.170598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.174037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.175902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.177693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 66/124] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:23 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:24.178480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.198160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.199198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.199678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.200206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.212131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:24.218338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:24.219227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:24.220037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:24.222496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 59/125] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:24.223472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.224793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.338085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7265-c1b5-4cf2-8870-637ade14a5c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.349020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.385010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.385693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.385980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.386256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/59fd3c40e3254e4abe88a9801df7949d/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.388614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, user_id=59fd3c40e3254e4abe88a9801df7949d, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.390645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.396716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547b0c-8e8b-42d5-9020-b7633cbd4aab None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.420997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 60/126] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:24 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/59fd3c40e3254e4abe88a9801df7949d/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:24.420997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.466263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.468006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.468336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.468634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.471397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.473299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.478548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 67/127] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:24.479226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.532966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.620112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c0fa-bcab-421d-bb10-2f1c1d16784b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.669492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.670486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.670957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579365f8-91d1-4b0c-ba38-e7ad484f7a95 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.671196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.671668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.672812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:24.673348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.675881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.687264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:24.689362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.703584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.718648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.719339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.719626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.720420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.733431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:24.740082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:24.740965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:24.741806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 68/129] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:24.743927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.826040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779032b0-c489-4763-a5a9-974acd2c95b4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.842382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.871284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.872511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.873134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.873604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/3cc351f3cc8d45098e84aaabaeda36e6/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.875995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, user_id=3cc351f3cc8d45098e84aaabaeda36e6, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.877834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.882263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd82920-7e5d-4dc5-a14a-c268360f3de3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.897508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 62/130] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:24 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/3cc351f3cc8d45098e84aaabaeda36e6/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:24.898879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.934850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.935395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.935693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.936220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.937707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.939427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.942264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 69/131] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:24.950585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:25.081967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d91c0d-e8a7-4739-ba24-e0f0b92f9ba6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.095178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:25.141944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.143025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.143758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.144236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.145200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:25.145775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.147402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.164111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 63/132] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:25.170162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:25.192635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df5281-7574-478b-9a40-35037ca52e48 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.210873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:25.256105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.257157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.257157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.257316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] PATH_INFO: `/v3/projects/d3e13b1664054eb1ad670422685b6ba8` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.258495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] RBAC: Authorizing `identity:get_project(project_id=d3e13b1664054eb1ad670422685b6ba8)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.260020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.262247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 70/133] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/projects/d3e13b1664054eb1ad670422685b6ba8 => generated 312 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:25.268140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:25.295137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d504d-4e83-4a7d-81e0-4c71cf462caf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.334552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.335678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.336153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.336666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.341662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.345561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.349655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 64/134] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:25 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:25.356014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:25.466588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac457-b2ab-48da-9eec-bf1bf586df66 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.493056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:25.504152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.504950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.505255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.505539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] PATH_INFO: `/v3/projects/d3e13b1664054eb1ad670422685b6ba8/users/59fd3c40e3254e4abe88a9801df7949d/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.507591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3e13b1664054eb1ad670422685b6ba8, user_id=59fd3c40e3254e4abe88a9801df7949d, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.509114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39903231-8a05-4d99-90ad-148eb7b85296 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.516355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f612-3e9b-48da-b488-d3f0f4c0226f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.525369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:25.546796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 71/135] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:25 2022] PUT /identity/v3/projects/d3e13b1664054eb1ad670422685b6ba8/users/59fd3c40e3254e4abe88a9801df7949d/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:20:25.548982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141c0d-86ec-4650-8405-1dee05c6102d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.549448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141c0d-86ec-4650-8405-1dee05c6102d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.549926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2141c0d-86ec-4650-8405-1dee05c6102d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.560529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.561321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.561607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.561931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.563179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.565412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.567087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 65/136] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:25 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:25.569550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 72/137] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:25 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:25.574144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:25.575645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24768-d784-4544-bb82-4aadfda5b2a7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.575939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24768-d784-4544-bb82-4aadfda5b2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.576221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24768-d784-4544-bb82-4aadfda5b2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.625692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-00b24768-d784-4544-bb82-4aadfda5b2a7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:25.627691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00b24768-d784-4544-bb82-4aadfda5b2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:25.694803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287efdf7-0a7f-4c69-91c8-0f55f76c6334 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:25.740435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:25.741067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.741348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.742645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/59fd3c40e3254e4abe88a9801df7949d/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.747520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, user_id=59fd3c40e3254e4abe88a9801df7949d, role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:25.749043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d3b1-7159-44fa-9f99-f918a4aa7905 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:25.776822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 66/138] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:25 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/59fd3c40e3254e4abe88a9801df7949d/roles/2be7d12a9a644f299290473830d5c537 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:25.783528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629b5a-e6c7-41d5-bec8-65c955300a75 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.783910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629b5a-e6c7-41d5-bec8-65c955300a75 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.784370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 73/139] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:25.784718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629b5a-e6c7-41d5-bec8-65c955300a75 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.786664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 67/140] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:25.792231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:25.792770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:25.793263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:25.823299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:25.824835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:25.929749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 74/141] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:26.118433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629b5a-e6c7-41d5-bec8-65c955300a75 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:26.158079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:26.266338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629b5a-e6c7-41d5-bec8-65c955300a75 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:26.313518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:26.314785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:26.315076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:26.315355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:26.319618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:26.321173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:26.322941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 68/142] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:26.328499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:26.412137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:26.459429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:26.489792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9876a8-2c1e-4097-965d-09c1134eca93 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:26.555739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:26.556620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:26.559879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:26.560317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:26.561160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:26.562779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:26.604022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 75/143] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:26 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:26.665052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:26.673884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64597bd3-80d6-4c78-a446-2810ed1dbc49 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:26.731903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:26.732569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:26.732883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:26.733158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:26.745587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:26.750825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:26.751816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:26.752688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 69/144] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:26.762541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:26.935170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:26.943832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f2600-2583-4025-9171-39ffa7189a4e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:27.003276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:27.004145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.004432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.004733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.005420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:27.005729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:27.007124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:27.034089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 76/145] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:27.042920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:27.175838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc7251-3a93-4115-a0a5-d25b7a82f86e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:27.216190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:27.216788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.217069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.217344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] PATH_INFO: `/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.218491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] RBAC: Authorizing `identity:get_user(user_id=fe4e43e5efba467bab6c9cd3d18ca28f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:27.220671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd224-4196-4e13-8f14-c756fec1d5bd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:27.222354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 70/146] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:27 2022] GET /identity/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:27.229421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:27.365699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e1ab2-6aa3-4d58-9e75-5dc14a493c38 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:27.383029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1b8b-4d8a-4db4-916f-6d11fdc73dbc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.383647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1b8b-4d8a-4db4-916f-6d11fdc73dbc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.383970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba1b8b-4d8a-4db4-916f-6d11fdc73dbc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.385715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 71/147] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:27.391357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0462-e0be-4656-a0d5-efe1325e445c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.391645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0462-e0be-4656-a0d5-efe1325e445c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.392489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0462-e0be-4656-a0d5-efe1325e445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.437843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:27.438642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.438922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.439197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.440798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:27.442317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:27.444154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 77/148] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:27 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:27.445061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-70bc0462-e0be-4656-a0d5-efe1325e445c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:27.446430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70bc0462-e0be-4656-a0d5-efe1325e445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:27.451769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:27.491296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:27.508515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:27.582315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 72/149] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:27.616148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021e5d1-a81e-4b08-b283-1314664194e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:27.653600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:27.654268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.654546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.654819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/fe4e43e5efba467bab6c9cd3d18ca28f/roles/a5028fb57d8d4c1ea4f83d33bc1454d5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.656804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, user_id=fe4e43e5efba467bab6c9cd3d18ca28f, role_id=a5028fb57d8d4c1ea4f83d33bc1454d5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:27.659224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae7084-0c7e-4f37-8ee1-7936d418e4f8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:27.683579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 78/150] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:27 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/users/fe4e43e5efba467bab6c9cd3d18ca28f/roles/a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:27.786867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264ad9-df2d-4254-8840-f03967e0f9a3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.787207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264ad9-df2d-4254-8840-f03967e0f9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.787455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264ad9-df2d-4254-8840-f03967e0f9a3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.790875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 73/151] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:27.797527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.797824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.798118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.822956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:27.824178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:27.914466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 79/152] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:28.005495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c264ad9-df2d-4254-8840-f03967e0f9a3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.154032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c264ad9-df2d-4254-8840-f03967e0f9a3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.200286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.200970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.201255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.201531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.218672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:28.224640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:28.225540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:28.226318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 74/153] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:28.237966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.266684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.270000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:28.298452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:28.380009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eeb4c-05e9-4333-8a4c-a2b81cd0d0be None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.418663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.419201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.419480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.419752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.420427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:28.420769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:28.422293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:28.432266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 80/154] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:28.438185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.509866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a2e4d-6c62-4842-8caa-cbee4d39bc46 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.529152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:28.550781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b4509-03b5-4515-b5fb-99212b16389e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.564569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.566904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.567418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.567894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.579469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:28.589504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:28.590476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:28.591533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:28.592248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.592248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.592248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.592248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.592521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:28.593980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:28.595040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 75/155] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:28.600606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.601481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 81/156] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:28.608088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.716706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:28.722760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689d5e0-ad9d-454e-b1ee-77c286db4c89 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.750161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194961dd-717e-4026-b79f-b5590e4353a0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:28.760287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.761111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.761486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.761852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.811067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:28.812267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:28.813083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:28.813503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:28.814480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:28.815282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:28.815706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:28.817275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:28.830633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 76/157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:28.853663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:28.854740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:28.858296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:29.039326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68039f81-b87e-45eb-a00d-4dd4bb7394e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:29.091177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 82/158] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:29.097915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:29.106354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:29.107278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.107704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.108133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] PATH_INFO: `/v3/users/3cc351f3cc8d45098e84aaabaeda36e6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.109509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] RBAC: Authorizing `identity:get_user(user_id=3cc351f3cc8d45098e84aaabaeda36e6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:29.111721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:29.117679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 77/159] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:28 2022] GET /identity/v3/users/3cc351f3cc8d45098e84aaabaeda36e6 => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:29.124326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:29.138674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:29.212973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:29.335090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69954e-cc96-436d-9ae0-cad83b84327d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:29.387312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43566a5-213e-4434-a89c-73ce41c65a57 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:29.457533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:29.458727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.459099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.459435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.466646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:29.472381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4edca0-bca6-4113-a822-a081266aac10 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:29.478554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 83/160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:29 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:29.482271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:29.483358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.483921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.484383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.485132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebefc3-b857-40c0-9f43-7477ddbd511f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.485844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebefc3-b857-40c0-9f43-7477ddbd511f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.486425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebefc3-b857-40c0-9f43-7477ddbd511f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.494929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 84/161] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:29.494929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebefc3-b857-40c0-9f43-7477ddbd511f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:29.654776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:29.656440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68136e07-7cc1-4a41-a489-739b73f9ee63 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:29.677173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:29.806508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebefc3-b857-40c0-9f43-7477ddbd511f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:29.868998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:29.955919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:29.956909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.957337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.957620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.958456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:29.958967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:29.960401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5a10e-b9d9-4917-9a23-d8f0a9c4f0b8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:29.970679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 78/162] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:20:29.973599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60006e9-76b5-4b6a-8d49-d3336b142e79 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.974355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60006e9-76b5-4b6a-8d49-d3336b142e79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.974997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60006e9-76b5-4b6a-8d49-d3336b142e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.988468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 85/163] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:29 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 608 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:29.990245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dde38f-207e-43fb-9b3a-1cb8bf6bedc1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.990638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dde38f-207e-43fb-9b3a-1cb8bf6bedc1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.991012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dde38f-207e-43fb-9b3a-1cb8bf6bedc1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.000172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f60006e9-76b5-4b6a-8d49-d3336b142e79 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:30.000371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 86/164] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:30.002198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dde38f-207e-43fb-9b3a-1cb8bf6bedc1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.003145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f60006e9-76b5-4b6a-8d49-d3336b142e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:30.104402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 79/165] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:30.106135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:30.106677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:30.107160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.136288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:30.138123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:30.175021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:30.227138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 80/166] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:30.286420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dde38f-207e-43fb-9b3a-1cb8bf6bedc1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:30.345919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:30.346586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:30.346870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:30.347144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.348395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:30.350530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:30.353514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 87/167] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:30.359589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.556812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57867-c22b-43f1-afad-36622f0ac512 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:30.625391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:30.626538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:30.626829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:30.627096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.627839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=3cc351f3cc8d45098e84aaabaeda36e6) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:30.628144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:30.629616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:30.637117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 81/168] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/role_assignments?user.id=3cc351f3cc8d45098e84aaabaeda36e6&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:30.658092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.737166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:30.796482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e360d0d-64b3-4437-8e0d-6912dedf5af9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:30.843521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:30.844082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:30.844359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:30.844648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.870088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:30.876661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:30.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:30.881206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:30.882029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:30.882642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:30.890774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.967701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:31.184953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543230-30ba-4e31-b383-0c747d007d7b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.207836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052db0e-77cd-44e7-a7cf-f497bc475d55 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.208442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:31.284064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:31.285117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.285603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.286075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.287273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:31.287798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:31.290394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:31.319060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 89/170] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:31.320118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:31.321243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.321720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.322181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.330373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:31.362895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:31.375876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:31.376900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:31.377991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:31.380104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 82/171] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:31.389367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:31.556459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492efed-cb9d-4a4c-bb85-a8efeee836b3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.569441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:31.588729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9747650-5a7c-48df-8b52-01ccb50cb1f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.619289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:31.620728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.621403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.622047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.624135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:31.627427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9954627-e60b-4f1e-ac0b-64efb3eaca36 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:31.630564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 90/172] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:31 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:31.632944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9857-edd9-454d-8a38-b40ba82a6865 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.633681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9857-edd9-454d-8a38-b40ba82a6865 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.634392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9857-edd9-454d-8a38-b40ba82a6865 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.637254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 91/173] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:31.638834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad9857-edd9-454d-8a38-b40ba82a6865 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:31.649420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:31.650061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.650333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.650777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.651614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:31.652070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:31.653685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c312471-3f7b-4afd-84d1-8a95d69e1aeb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:31.667419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 83/174] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:31.668986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821c32-37b9-4fbd-aa36-8947a7f9dd53 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.669426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821c32-37b9-4fbd-aa36-8947a7f9dd53 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.669836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821c32-37b9-4fbd-aa36-8947a7f9dd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.699035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ae821c32-37b9-4fbd-aa36-8947a7f9dd53 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:31.700257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ae821c32-37b9-4fbd-aa36-8947a7f9dd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:31.716759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:31.788781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 84/175] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:31.790350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c972-c28e-4d71-9436-9eb8241f0139 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.790662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c972-c28e-4d71-9436-9eb8241f0139 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.790908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808c972-c28e-4d71-9436-9eb8241f0139 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.792152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 85/176] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:31.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808c972-c28e-4d71-9436-9eb8241f0139 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:31.818329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad9857-edd9-454d-8a38-b40ba82a6865 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.868215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:31.869192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:31.869725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:31.870305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:31.958120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:31.989423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:31.991286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808c972-c28e-4d71-9436-9eb8241f0139 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:31.992407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:32.052741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.053665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.054139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.054574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] PATH_INFO: `/v3/users/59fd3c40e3254e4abe88a9801df7949d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.055864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] RBAC: Authorizing `identity:get_user(user_id=59fd3c40e3254e4abe88a9801df7949d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:32.057569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fd27c-5e82-44b8-ac6a-e7df2fd2aa73 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:32.059479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 86/177] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:31 2022] GET /identity/v3/users/59fd3c40e3254e4abe88a9801df7949d => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:32.064856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.066285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f51d3-5f27-4866-ae73-ad375c032e51 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.066586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f51d3-5f27-4866-ae73-ad375c032e51 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.066849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f51d3-5f27-4866-ae73-ad375c032e51 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.068502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 87/178] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:32.069402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.069680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.069950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.095455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:32.096701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:32.156520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 92/179] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:32.158534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.187664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 88/180] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:32.188365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.335080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.348560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb50152-d2b5-4102-a942-c9e01c02381d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:32.376197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48a1c7-bf8c-4f1a-9df3-5ef7e753fed3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:32.405381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.406025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.406302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.406573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.415905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.419484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7b60066a-cfbb-4c92-b8d9-9c014ff254ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:32.425121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:32.426055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:32.426890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:32.427537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.427537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.427537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.427537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.428555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 93/181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:32.433856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.434318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.434594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.453289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:32.455194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:32.515085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:32.516643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773764f0-ec80-4ed3-8a65-fdc9bba802a6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:32.535237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 94/182] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:32.536117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.615872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.644362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 89/183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:32.644362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e8f3-66c0-4e3e-86bc-3f3656692aac None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.644362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e8f3-66c0-4e3e-86bc-3f3656692aac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.644362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e8f3-66c0-4e3e-86bc-3f3656692aac None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.644362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 90/184] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:32.646110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0e8f3-66c0-4e3e-86bc-3f3656692aac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.681453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a5358-f197-4800-bda2-bfe11bfe8e11 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:32.724916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.731821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.732474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.732814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.733095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.734319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:32.736505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:32.738234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 95/185] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:32.740633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.796046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0e8f3-66c0-4e3e-86bc-3f3656692aac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:32.834759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.835399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.835675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.835943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.836664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:32.836972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:32.838384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:32.851001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 91/186] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:32.851001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.876578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1adc14-8639-4803-829d-951143d9902e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:32.906020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.915268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:32.915913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.916189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.916456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.927330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-68cdee8b-3596-4f06-b53d-3d4b8dbb7529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:32.932705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:32.933804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:32.934617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 96/187] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:32.935276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.935276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.935506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.956143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:32.957412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:32.979827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afaf11-c2ea-40ba-b330-005cfbd716ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.018528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.019380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.020030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.020314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.020598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] PATH_INFO: `/v3/projects/d3e13b1664054eb1ad670422685b6ba8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.021964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] RBAC: Authorizing `identity:get_project(project_id=d3e13b1664054eb1ad670422685b6ba8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.023483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.025262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 92/188] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/projects/d3e13b1664054eb1ad670422685b6ba8 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.027481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.042487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 97/189] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:33.043342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.160169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b2aa-f294-4be1-8597-83fa75e3bb97 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.178065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18931369-f93b-4a54-bc5b-9c1b35c5e74e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.178670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.209337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.210102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.210407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.210543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.211551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:33.212074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.213414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.221549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 93/190] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=2be7d12a9a644f299290473830d5c537 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.223812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.224550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.224794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.225143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.234319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.239999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:33.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:33.246507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:33.247650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 98/191] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:33.255305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.328210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.384057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062e352-278a-456f-a797-29c6d2bd3dfb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.397019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68175d-599b-43cc-ac79-5c0debe4b49e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.436352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.437428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.437829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.438292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] PATH_INFO: `/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.440064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] RBAC: Authorizing `identity:get_user(user_id=fe4e43e5efba467bab6c9cd3d18ca28f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.441007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.441645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.441923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.442200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.442556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.442889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:33.444244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.447166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 94/192] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/users/fe4e43e5efba467bab6c9cd3d18ca28f => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.447316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.447410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.459608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 99/193] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.460568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.510591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.576865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae68dc8-35d0-4eca-abaf-696a35d6db88 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.579251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16dbf9-2d65-4696-b73f-f21009943170 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.591405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.616159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.616954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.617618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.617897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.618648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=d3e13b1664054eb1ad670422685b6ba8, user.id=59fd3c40e3254e4abe88a9801df7949d) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:33.618953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.620439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.627056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 95/194] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/role_assignments?user.id=59fd3c40e3254e4abe88a9801df7949d&scope.project.id=d3e13b1664054eb1ad670422685b6ba8&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.627573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.627796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.628517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.628927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.629267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.630317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:33.630716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.632914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.646212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 100/195] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.651850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.787716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28ddab-5f0f-4135-8c1e-c7393edb666f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.799720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.805789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b262-7ead-43ec-9147-b5c461c1688b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.841190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.841862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.842161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.842445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.843664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.845808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.847475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 96/196] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:33.849678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:33.864919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.864919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.865247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.865554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.921398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.950781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.952281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.975589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a9cf9-258c-4ec0-a501-56593aa2f734 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.009979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.010630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.010886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.011159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.027994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.029513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.031216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 97/197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:34.031996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.061030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 101/198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:34.065878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.066010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.142808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.160437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b727c-ada7-4276-a85f-2860977cfaa9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.181217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e96ae-9561-4e3b-8812-78132a161562 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.197681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.198338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.198620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.198898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.200060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.201545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.203175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 98/199] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:34.203958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.228361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.229027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.229309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.229584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.230253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.231771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.258747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 102/200] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:34 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:34.259667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.336107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.339800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ef63-c720-4f08-866d-843281b296a8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.383374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.384019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.384359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.384690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.399223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc339a-4d3e-4b19-ad0b-45c4f8333f38 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.439280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.452695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.455320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.455320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.455320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.456842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.458374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.460371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 103/201] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:34.461122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.478647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.480404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.617800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408a94-8ba4-48ea-805f-5f5f88d0979e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.644181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.656832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 99/202] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:34.656832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.669799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.670773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.671254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.671729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.739608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:34.747275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.749143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.843979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be3ba4-ce3b-4e0e-b09e-5bf20f54051f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:34.882927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 104/203] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:34.885298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:34.915706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:34.916757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:34.917242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:34.917719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:34.919691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77, user.id=fe4e43e5efba467bab6c9cd3d18ca28f) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:34.920199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:34.923963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:34.938882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 100/204] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/role_assignments?user.id=fe4e43e5efba467bab6c9cd3d18ca28f&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:34.940854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:35.085002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:35.115209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:35.136553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf41d2-f87a-40cb-bd86-4b636095f0c6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:35.183226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d0cd-af94-4156-a0b4-91bf5ea01eb4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:35.196516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:35.198076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.198752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.199398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.201437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:35.203941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:35.207533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 105/205] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:35.213960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:35.251287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:35.252297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.252779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.253211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.255579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:35.258774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:35.262529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 101/206] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:34 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:35.272633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:35.384889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93508b3-8f9d-444b-b687-90f9052edb09 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:35.421137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff6ea-fdb3-4c65-b183-e993baf39ddc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:35.446509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:35.447801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.448608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.449430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.451092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=f066de39601f400db71eaa6e3c4c356b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:35.451838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:35.455216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d20c15-89c1-448e-b14d-57c9654699f6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:35.457034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:35.463144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:35.463769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.464053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.464334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.465134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=f066de39601f400db71eaa6e3c4c356b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:35.465447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:35.466932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47eca73-bb25-43ab-9b94-81603fbe11f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:35.467411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 106/207] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:35 2022] GET /identity/v3/role_assignments?user.id=f066de39601f400db71eaa6e3c4c356b&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:35.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 102/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:35 2022] GET /identity/v3/role_assignments?user.id=f066de39601f400db71eaa6e3c4c356b&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=2be7d12a9a644f299290473830d5c537 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:36.662633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c719-be26-4d24-a1e7-f23dfb4185ef None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:36.663152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c719-be26-4d24-a1e7-f23dfb4185ef None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:36.663152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3c719-be26-4d24-a1e7-f23dfb4185ef None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:36.664678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 107/209] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:36.670015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c432-b1d3-4118-aea9-e14b3d5cbe76 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:36.670871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c432-b1d3-4118-aea9-e14b3d5cbe76 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:36.671346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720c432-b1d3-4118-aea9-e14b3d5cbe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:36.691235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6720c432-b1d3-4118-aea9-e14b3d5cbe76 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:36.692916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6720c432-b1d3-4118-aea9-e14b3d5cbe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:36.767240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 103/210] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:36.857258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6a0d-6539-4834-a3d5-8ee6dfeb7429 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:36.858237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6a0d-6539-4834-a3d5-8ee6dfeb7429 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:36.859035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6a0d-6539-4834-a3d5-8ee6dfeb7429 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:36.862316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 108/211] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:36.871248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:36.871835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:36.872190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:36.893771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:36.893771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:36.966079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 104/212] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:37.063136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6a0d-6539-4834-a3d5-8ee6dfeb7429 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.102778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:37.194357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6a0d-6539-4834-a3d5-8ee6dfeb7429 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:37.233259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:37.233820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.234102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.234368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.236064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:37.237564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:37.239173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 109/213] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:37.244903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.253670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.325117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:37.366608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:37.367112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ecfe0-14c2-4829-b7e1-8edbce689078 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:37.372923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef664d2-0a89-4353-b895-bb86f17fe518 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:37.406940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:37.408661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.409110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.409547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.409784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:37.410482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.410834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.411180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.411340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:37.413153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f4b7b-f03e-4432-aebf-21efa9d8f47a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:37.422213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:37.428281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:37.429364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:37.430249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:37.431010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 110/214] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:37.437202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.452553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 105/215] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:37 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:37.506599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60b88-7a4b-40e5-94a8-1408ce9097a2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.507285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60b88-7a4b-40e5-94a8-1408ce9097a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.507733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60b88-7a4b-40e5-94a8-1408ce9097a2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.509782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 106/216] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:37.519232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57dd7d-ff15-4c4a-bafd-2d9c6968dc30 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.519792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57dd7d-ff15-4c4a-bafd-2d9c6968dc30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.520294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57dd7d-ff15-4c4a-bafd-2d9c6968dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.541784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2e57dd7d-ff15-4c4a-bafd-2d9c6968dc30 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:37.543344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2e57dd7d-ff15-4c4a-bafd-2d9c6968dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:37.590625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f523d4-5356-4e66-a026-20574d445615 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:37.626678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 107/217] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:37.628219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7a32-c48b-4320-861b-86cbc2c3d52b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.628502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7a32-c48b-4320-861b-86cbc2c3d52b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.628887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7a32-c48b-4320-861b-86cbc2c3d52b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.630285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 108/218] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:37.635970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.636346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.637977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.638084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:37.638084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.638084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.638084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.639802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:37.639802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:37.640310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:37.652207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 111/219] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:37.657912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:37.658948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.659275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:37.678872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:37.701104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:37.749885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 109/220] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:37.784834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f5f3-6258-45f9-8ea1-7e829e3f2c97 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:37.822881 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:37.823744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:37.824195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:37.824466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] PATH_INFO: `/v3/users/8b4ecf01e4b441af9309a038374339c9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:37.845550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] RBAC: Authorizing `identity:get_user(user_id=8b4ecf01e4b441af9309a038374339c9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:37.848303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:37.852095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 112/221] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/users/8b4ecf01e4b441af9309a038374339c9 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:37.859421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.912398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:37.986153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731dc47-1505-4593-8e2f-e835f9d146eb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.016336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.036052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.036784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.037156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.037579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.045454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b7056-780f-4e9e-9229-9f6b6699c5ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.088084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.088756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.089038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.089314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.096425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.101165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:38.106244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:38.107348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:38.108254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 113/222] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:38.109172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.126134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.128705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.228358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b96ba4-c4d7-450d-8d0a-5d1a565cf3d6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.264785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 110/223] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:38.265848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.276037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.276738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.277016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.277284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.289151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:38.294662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:38.295571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:38.296421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 114/224] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:38.297118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.333138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.349308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.382544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afc369-78ec-4fac-87ae-61f3b09448d9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.405086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85783e-be41-45e4-b314-4a0e7b72276a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.420411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.421282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.421703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.422113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.422979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:38.423419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.424968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.434396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 111/225] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.435443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.449753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.450563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.450946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.451319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.452632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.454760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.461890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 115/226] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.462956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.533676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.549457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe1a33-553a-4254-8c97-375ef903172e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.575924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea43d27-93da-4944-80dd-7c2786299109 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.586766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.587590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.588011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.588446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.589279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:38.589719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.591219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.592775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.606929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 112/227] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.607978 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.617301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.618340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.618475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.618858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.620127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.621715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.628996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 116/228] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.629866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.723577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268d6e-51de-44e7-b639-e80b527afdc7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.740051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d1a07-2db5-4a1e-8ee5-8c9406ba1667 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.761199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.762041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.762464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.762882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.763815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=8b4ecf01e4b441af9309a038374339c9) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:38.764298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.766094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.772868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 113/229] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/role_assignments?user.id=8b4ecf01e4b441af9309a038374339c9&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.773900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.783147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.791780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.792482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.792858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.793205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.794468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.796121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.798471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 117/230] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:38.806790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.889484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.972627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadaad0-ea91-4db6-ba5d-3a5b8ce739f0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:39.015415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807435-2a37-417f-bc20-a41338b0cd2a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:39.023438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:39.024167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.024522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.024909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.026112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:39.028176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:39.030163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 114/231] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:39.037304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:39.105748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:39.106377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.106663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.106940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.108091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:39.109686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:39.112406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 118/232] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:39.119373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:39.161404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e75a6-e032-4ea5-90ec-a110bdf5610b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:39.192949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:39.204729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:39.205650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.205650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.206411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/f066de39601f400db71eaa6e3c4c356b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.209039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, user_id=f066de39601f400db71eaa6e3c4c356b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:39.210507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8aa199-df1e-450f-82f1-23fbbffb9656 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:39.234956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 115/233] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:39 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/f066de39601f400db71eaa6e3c4c356b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:20:39.250558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:39.290257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4ae7d-cab7-4a91-b7e2-1a0bc43c74d0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:39.346116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:39.347224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.347695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.348158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/f066de39601f400db71eaa6e3c4c356b/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.351741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, user_id=f066de39601f400db71eaa6e3c4c356b, role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:39.353877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745feccb-0d20-4fde-9635-ba901120fa23 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:39.380934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 119/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:39 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/f066de39601f400db71eaa6e3c4c356b/roles/2be7d12a9a644f299290473830d5c537 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:39.957098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eff76-805a-49cc-a1c8-93e784605ffd None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.957611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eff76-805a-49cc-a1c8-93e784605ffd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.957831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543eff76-805a-49cc-a1c8-93e784605ffd None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.959096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 116/235] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:39.973152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.973647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.973647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:39.997749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:39.998470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ad9f-3c97-42ab-9748-9e564a7eef07 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:39.998754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ad9f-3c97-42ab-9748-9e564a7eef07 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:39.998966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:39.999071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0ad9f-3c97-42ab-9748-9e564a7eef07 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.000857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 117/236] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:40.006813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.007094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.007368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.038044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:40.039241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:40.103202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 120/237] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:40.128773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 118/238] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:40.456253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:40.497084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:40.530785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:40.616512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68876c1f-201c-4155-8cf8-9af8cf53ad44 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:40.624669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e4949-5657-40f3-adcd-253fefad2996 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:40.650558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:40.677515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:40.678160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.678436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.678705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.690114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:40.693879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:40.694527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.694802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.695070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:40.695181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:40.696012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:40.696875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:40.700694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 121/239] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:40.701605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:40.702432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:40.702807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:40.705271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 119/240] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:40.729723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:40.827269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f2357-6af8-42e0-a273-7e3d019ade02 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:40.849699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:40.882149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:40.882883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.883165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.883441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.884120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:40.884422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:40.885974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:40.902487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 122/241] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:40.910927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:40.922247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6f4b9-4699-4201-a2b6-b50f81b3c71e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:40.973398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:40.974075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.974317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.974599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.975272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:40.976777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.011612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 120/242] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:40 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:41.040301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1818e-62c6-428e-b494-8fad36b1a2ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.077231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.077868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.078143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.078413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] PATH_INFO: `/v3/users/73971d7be9d449c19815fbd4c4d4a08e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.092854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] RBAC: Authorizing `identity:get_user(user_id=73971d7be9d449c19815fbd4c4d4a08e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:41.094347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af7ad3-9867-439c-a717-87390574aa11 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.096929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 123/243] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:40 2022] GET /identity/v3/users/73971d7be9d449c19815fbd4c4d4a08e => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:41.103197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:41.193161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.283834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5062e-4027-4591-9466-3e6b64f97ae3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.331272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.331891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.332171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.332446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.361759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989c7ce-a7d7-4dfc-9b27-2702e0fd0e38 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.362071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989c7ce-a7d7-4dfc-9b27-2702e0fd0e38 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.362315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989c7ce-a7d7-4dfc-9b27-2702e0fd0e38 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.364040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 124/244] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:41.370491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.370774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.371039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.392956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:41.394169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:41.428230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:41.430234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.431021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.472795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 125/245] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:41.487277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.561109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 121/246] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:41.565589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:41.686471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c1666-2f2f-4382-ac9a-099178d55aaa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.727472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.728487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.728988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.729458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.730780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:41.732438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.734625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 126/247] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:41 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:41.740952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:41.799509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:41.845563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.888161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafa85d-4d45-47a6-8d46-343d22a58844 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.927393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd666234-bc84-4e61-94c3-bb4bd1f36764 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.935578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.936206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.936491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.936794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.938021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=73971d7be9d449c19815fbd4c4d4a08e) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:41.938548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:41.940528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d97a6-bd39-47cd-b50a-c96cc25b98f5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.948296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 122/248] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:41 2022] GET /identity/v3/role_assignments?user.id=73971d7be9d449c19815fbd4c4d4a08e&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:41.949881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b3b99-d86f-471c-9f1f-04cc2fa89ebe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.950326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b3b99-d86f-471c-9f1f-04cc2fa89ebe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.951052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b3b99-d86f-471c-9f1f-04cc2fa89ebe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.953135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 123/249] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:41.959495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102da53-c7d2-4759-b2fa-6b59256e56ea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.960118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102da53-c7d2-4759-b2fa-6b59256e56ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.960775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102da53-c7d2-4759-b2fa-6b59256e56ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.980043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.982316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.983283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.983764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.984800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.988028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6102da53-c7d2-4759-b2fa-6b59256e56ea None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:41.989716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6102da53-c7d2-4759-b2fa-6b59256e56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:42.003169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4b5cbb60-38b6-4e6a-b9dd-3dc3fb49e84f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:42.009008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:42.010056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:42.010896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:42.013467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 127/250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:42.019064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566c4a-84fc-42b5-8355-4e01b0f80c9d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.019603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566c4a-84fc-42b5-8355-4e01b0f80c9d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.020121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566c4a-84fc-42b5-8355-4e01b0f80c9d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.022110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 128/251] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:42.025567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30566c4a-84fc-42b5-8355-4e01b0f80c9d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.076821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 124/252] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:42.078597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.079152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.079672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.109814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:42.111720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:42.155655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:42.169872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30566c4a-84fc-42b5-8355-4e01b0f80c9d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:42.208475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:42.209407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.209843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.210254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.211080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:42.211523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:42.213196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:42.222717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 129/253] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:42.230507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.246946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 125/254] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:42.374622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.375209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690a898-c20a-4a19-b34d-377a747ed9ae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:42.462929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:42.463885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.464306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.464746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] PATH_INFO: `/v3/users/8b4ecf01e4b441af9309a038374339c9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.466682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] RBAC: Authorizing `identity:get_user(user_id=8b4ecf01e4b441af9309a038374339c9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:42.468389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:42.470728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 130/255] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/users/8b4ecf01e4b441af9309a038374339c9 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:42.477420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.480481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:42.538535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09c266-b4b2-408f-a0e9-ad02a037c16e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:42.538882 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:42.574732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:42.575379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.575805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.576082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.587149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:42.593012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:42.593908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:42.596573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 126/256] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:42.600663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.732469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f559-cfa9-4a44-ad52-4fbc6fae1cbd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:42.794503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae75708-7e8c-43ba-b500-b9fcdb6a92e6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:42.804725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:42.805705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.806173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.806657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.808056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:42.809755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:42.811695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 131/257] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:42.812828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.866948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:42.888136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:42.888781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:42.889058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:42.889324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:42.889998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:42.890296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:42.892276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:42.908383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 127/258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:42.909300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:42.997650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.011500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf270b4-8add-43da-bed7-8a105e311de3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.080482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.081234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.081513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.081781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.095582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a7b0-d55b-4dee-9fb2-5a44fe4a3da1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.105018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:43.111164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:43.112085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:43.112912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:43.114652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 132/259] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:43.116674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:43.160188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.161065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.161523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.161938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/8b4ecf01e4b441af9309a038374339c9/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.167198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, user_id=8b4ecf01e4b441af9309a038374339c9, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:43.168727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:43.189368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 128/260] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:42 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/8b4ecf01e4b441af9309a038374339c9/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:43.195183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:43.315125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac8cd-5959-4344-8436-9ba809b7ba0b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.321342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.395101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.395761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.396040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.396310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.399587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:43.401253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:43.403678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 133/261] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:43 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:43.410355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:43.467270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274b49e-fe7b-42b9-9c42-70dfc06b6415 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.469058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.564621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.565350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.565585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.565819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.566494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:43.566795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:43.568198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:43.584009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 129/262] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:43.589825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:43.600825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1543d-951a-40e2-bde8-5641e69f6f86 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.643023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.643938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.644392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.644896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.735513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:43.737299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:43.784498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.813545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4f631-38d7-4ed2-b9aa-09da21fab49b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:43.829429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.885341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:43.886052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.886329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.886599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] PATH_INFO: `/v3/users/f066de39601f400db71eaa6e3c4c356b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.888067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] RBAC: Authorizing `identity:get_user(user_id=f066de39601f400db71eaa6e3c4c356b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:43.889568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:43.891174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 130/263] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:43 2022] GET /identity/v3/users/f066de39601f400db71eaa6e3c4c356b => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:43.900728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.019668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 134/264] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:20:44.025344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.122078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0172b25-3984-4949-ba1a-e968553a73ee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.187394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.188105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.188934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.189294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.256967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.264130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.265783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc44f-91f4-4e4f-8a13-e2c5ff529f52 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.270764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f3ebd-1c71-4303-bd9e-2a8edb4fa698 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.346976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.371279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 131/265] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:20:44.372877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0ce3d-7e33-4fa1-aa6f-b0efbc9e749d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.373197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0ce3d-7e33-4fa1-aa6f-b0efbc9e749d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.373565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0ce3d-7e33-4fa1-aa6f-b0efbc9e749d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.377892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 132/266] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:44.380467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0ce3d-7e33-4fa1-aa6f-b0efbc9e749d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.385308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.386167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.386589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.387005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.393365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.395131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29154d29-a03e-4e30-8ab5-46c693c59cd4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.397406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 135/267] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:44 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:44.398845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.399330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.399839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.449653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:44.451212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:44.484999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.493551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0ce3d-7e33-4fa1-aa6f-b0efbc9e749d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.531152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.531888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.532745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.533092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.534333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.536016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.538073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 133/268] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:44 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:44.539062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.618145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 136/269] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:44.619088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.670720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507da838-a725-4835-a3b6-474b9c4a2e43 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.727197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1c3dd-bf6d-49fb-a2c9-d8b9bd0d49e0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.739185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.739810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.740086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.740356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.741177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=f066de39601f400db71eaa6e3c4c356b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:44.741486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.742946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59692821-bf5d-4000-9297-d903311ccbaa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.749349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 134/270] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:44 2022] GET /identity/v3/role_assignments?user.id=f066de39601f400db71eaa6e3c4c356b&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=2be7d12a9a644f299290473830d5c537 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:44.751337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f2e20-3ab6-4341-9471-39216b168cb6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.751729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f2e20-3ab6-4341-9471-39216b168cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.752000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f2e20-3ab6-4341-9471-39216b168cb6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.753502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 135/271] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:44.760234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.760741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.761023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.761369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.794743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:44.795941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:44.798868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.799513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.799932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.800204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.800964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=f066de39601f400db71eaa6e3c4c356b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:44.801270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.802895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59990-f67d-437a-8228-b5eb52161ab2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.808986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.817675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 137/272] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:44 2022] GET /identity/v3/role_assignments?user.id=f066de39601f400db71eaa6e3c4c356b&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:44.823034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393bcf-9b67-4194-9b12-3ef1f47fc97d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.823697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393bcf-9b67-4194-9b12-3ef1f47fc97d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.824222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393bcf-9b67-4194-9b12-3ef1f47fc97d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.826347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 138/273] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:44.842802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.843444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.843921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.894546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:44.894546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:44.909765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 136/274] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:45.075295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.082400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 139/275] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:45.201506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81131d7e-d540-4edf-b20e-195b79929c56 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.240708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.241358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.241640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.241916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.250324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.255287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:45.262231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:45.263162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:45.264146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 137/276] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:45.273129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.394520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.399016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb8def-4e89-4f2d-aab3-bef76f099425 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.417562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.441071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.441753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.442032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.442308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.443161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:45.443534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:45.445077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:45.459614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 140/277] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:45.461056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.527249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e9fdb-2183-4c6a-b416-b3f5eed07501 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.564349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.565049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.565326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.565595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.578981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:45.584012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:45.585010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 138/278] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:45.586522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.606428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277058d-f176-48f1-9cb3-bc0f8bfec249 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.650718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.651300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.651974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.652234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.652497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] PATH_INFO: `/v3/users/59c09c1851274b2398542ed8b4088227` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.667897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] RBAC: Authorizing `identity:get_user(user_id=59c09c1851274b2398542ed8b4088227)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:45.669401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:45.671081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 141/279] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/users/59c09c1851274b2398542ed8b4088227 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:45.671842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.702682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662435e-89c8-4cff-bc85-19506c52c1ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.717439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.741811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.742728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.742984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.743259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.744860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:45.746320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:45.747897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 139/280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:45.747897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.796901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7acbe-4481-426d-b84e-6f705381cfc5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.836959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.837597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.837871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.838139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.838811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:45.839387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:45.840480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:45.850375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 142/281] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:45.853457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.872139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f187d77-3c8a-4beb-be5e-136ad83d4ffc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:45.910414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.923767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:45.924498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:45.924947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:45.925247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:45.926319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, user.id=59c09c1851274b2398542ed8b4088227, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:45.926761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:45.928813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:45.937910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 140/282] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/role_assignments?user.id=59c09c1851274b2398542ed8b4088227&scope.system=all&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:45.938799 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:45.991299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:45.998983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156f2c8-685b-4f6f-aba2-4114ec5b63c4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.043458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.044146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.044828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.045159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.045845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:46.047356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b6c1b-5821-4981-9d98-22aae9e65ba7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:46.072783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9f33f-708a-454d-b6e2-8463b03845e5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.073294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 143/283] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:20:45 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:46.074572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bec26-79d3-4657-9265-b055f408b432 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.074885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bec26-79d3-4657-9265-b055f408b432 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.075203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bec26-79d3-4657-9265-b055f408b432 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.076633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 144/284] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:46.085755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.086320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.086737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.121459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:46.122704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:46.135269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.137090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.137375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.137643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] PATH_INFO: `/v3/users/73971d7be9d449c19815fbd4c4d4a08e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.141922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] RBAC: Authorizing `identity:get_user(user_id=73971d7be9d449c19815fbd4c4d4a08e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:46.143426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:46.147632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 141/285] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:45 2022] GET /identity/v3/users/73971d7be9d449c19815fbd4c4d4a08e => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:46.152917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:46.232187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:46.239801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 145/286] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:46.286391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc08864-e7f8-4de7-8ea7-01552365303d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.326185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.326824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.327100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.327367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.328500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:46.330617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:46.332701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 142/287] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:46 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:46.338032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:46.435073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:46.458101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951ace-f839-4413-9653-c5b7894fe04d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.505559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.506510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.506873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.507285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/73971d7be9d449c19815fbd4c4d4a08e/roles/a5028fb57d8d4c1ea4f83d33bc1454d5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.510589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, user_id=73971d7be9d449c19815fbd4c4d4a08e, role_id=a5028fb57d8d4c1ea4f83d33bc1454d5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:46.512723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:46.521387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:46.532903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 146/288] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:20:46 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/users/73971d7be9d449c19815fbd4c4d4a08e/roles/a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:20:46.650606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5016620-55c6-47b0-bba5-2cf3c9e4b8d4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.685858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:46.695981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.696820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.697107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.697374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.714176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:46.719229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:46.720756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:46.721578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 143/289] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:46.728553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:46.842072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5dad-f924-42ca-b128-c25f6760ab25 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:46.881127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:46.881741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:46.882016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:46.882284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:46.882950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:46.883249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:46.884645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:46.894853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 147/290] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:46.900640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:47.031546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf148b-3ef1-45ab-950b-73933ade8c0f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:47.075021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14991-3493-475c-9554-98ff07550559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:47.075660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14991-3493-475c-9554-98ff07550559 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:47.075933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14991-3493-475c-9554-98ff07550559 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:47.076203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14991-3493-475c-9554-98ff07550559 None admin] PATH_INFO: `/v3/users/8b4ecf01e4b441af9309a038374339c9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:47.080525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b14991-3493-475c-9554-98ff07550559 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4ecf01e4b441af9309a038374339c9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:47.082201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b14991-3493-475c-9554-98ff07550559 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:47.083850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 144/291] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:46 2022] GET /identity/v3/users/8b4ecf01e4b441af9309a038374339c9 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:47.089735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:47.093914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:47.202958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8d637-03cf-4257-8250-3b3f23659a66 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:47.239243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:47.239870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:47.240150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:47.240421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:47.322305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:47.323890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:47.327170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:47.448118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 148/292] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:47.454806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14991-3493-475c-9554-98ff07550559 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:47.534301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:47.582678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14991-3493-475c-9554-98ff07550559 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:47.627020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:47.628027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:47.628527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:47.629041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:47.630405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:47.632048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df21c8-3f40-4f3f-a87e-a034f9bc4e0c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:47.634211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 145/293] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:47 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:47.645974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:47.753040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756553a-6074-41be-a0d1-fc1fdcc38128 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:47.755623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:47.791013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:47.791624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:47.791901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:47.792185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:47.792945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=8b4ecf01e4b441af9309a038374339c9) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:47.793249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:47.794795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46af0db-d739-4b5d-87e9-062e6774ab1f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:47.801808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 149/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:47 2022] GET /identity/v3/role_assignments?user.id=8b4ecf01e4b441af9309a038374339c9&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:48.462458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa31-ab66-4254-8faf-81b0fa670aed None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.463327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa31-ab66-4254-8faf-81b0fa670aed None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.463768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20aa31-ab66-4254-8faf-81b0fa670aed None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.465546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 146/295] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:48.471907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0d50-439b-484e-baeb-c709adc96bb6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.472868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0d50-439b-484e-baeb-c709adc96bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.473350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0d50-439b-484e-baeb-c709adc96bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.493217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7ddb0d50-439b-484e-baeb-c709adc96bb6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:48.494733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7ddb0d50-439b-484e-baeb-c709adc96bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:48.576048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 150/296] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:48.711345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ce39-c1bd-47f4-ae46-c2a7fd71c850 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.711656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ce39-c1bd-47f4-ae46-c2a7fd71c850 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.711880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ce39-c1bd-47f4-ae46-c2a7fd71c850 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.713645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 147/297] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:48.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04810-9fdc-4e4e-928c-4a1a81cdf5b6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.717355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04810-9fdc-4e4e-928c-4a1a81cdf5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.717592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04810-9fdc-4e4e-928c-4a1a81cdf5b6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.718749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.719034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.719254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 151/298] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:48.719446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.724458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:48.724831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:48.725055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:48.740867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:48.742309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:48.744701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:48.746030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:48.836610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 148/299] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:48.843396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 152/300] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:48.865796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:48.940101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:48.991989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db068828-5f0f-4158-b568-7459aeb28faf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.030055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.030697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.030977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.031248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-71a1570a-c2d5-4015-89d5-a6b6c40b2e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:49.052611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:49.053483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:49.054340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:49.057145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 149/301] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:49.062495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.099235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.122736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4830-7023-4174-a6df-da41e463a3a6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.123343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4830-7023-4174-a6df-da41e463a3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.123828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a4830-7023-4174-a6df-da41e463a3a6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.125997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 150/302] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:49.126964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a4830-7023-4174-a6df-da41e463a3a6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.175170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e334d-a89b-47e8-9991-7f832348b48d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.210896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.211509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.211788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.212060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.212756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:49.213059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.214439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be740a7-f392-466c-9da7-8f3cd6a3bcfa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.222899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 153/303] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:49.224041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.224382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.224770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.241231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a4830-7023-4174-a6df-da41e463a3a6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.242385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:49.243529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:49.255800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.276659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.277286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.277560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.277825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.289281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:49.294164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:49.295099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:49.296147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 151/304] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:49.300843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.315684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 154/305] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:49.316556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.340254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.422523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b13f1-cb3b-470a-a872-94422dc43048 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.425948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43051-2654-4b3a-8a4c-1090b4eed665 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.462863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.463512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.463794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.463902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.464317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] PATH_INFO: `/v3/users/59c09c1851274b2398542ed8b4088227` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.464512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.464824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.465092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.465258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] RBAC: Authorizing `identity:get_user(user_id=59c09c1851274b2398542ed8b4088227)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.466180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.466658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.467602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.473540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 155/306] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/users/59c09c1851274b2398542ed8b4088227 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:49.473540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.474128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 152/307] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:49.474762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.520180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.572490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.584491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6647c6-5791-4a8e-a3d4-24662b9375c8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.585666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea4e28-b504-4314-9271-60db5c9b6e4a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.622271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.623762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.623762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.623762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] PATH_INFO: `/v3/system/users/59c09c1851274b2398542ed8b4088227/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.625186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=59c09c1851274b2398542ed8b4088227, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.626755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.627295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.627472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.627819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.628149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.629032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:49.629395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.630825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.645916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 153/308] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 29 21:20:49 2022] PUT /identity/v3/system/users/59c09c1851274b2398542ed8b4088227/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:49.645916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.646207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 156/309] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:49.646207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.773845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a9d32-fb12-4be8-aa05-30706fe0ebd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.776883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74084503-bce5-4bb3-8631-cbea2383e6e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:49.783054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:49.818234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.818879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.819161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.819282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:49.819490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.819811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:49.820091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:49.820357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] PATH_INFO: `/v3/groups` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:49.821073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:49.822514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:49.832093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:49.837542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:49.838511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:49.845375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 157/310] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:49.846410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 154/311] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 29 21:20:49 2022] POST /identity/v3/groups => generated 229 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:49.846486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.846554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:49.865128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.004765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f8764-a6a7-4c48-b42f-3313662397c8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.005786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e66c-7e9f-48cd-9f0a-81897928f47b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.047435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.048388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.048506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.048984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.049277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.049554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.049661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.049846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] PATH_INFO: `/v3/users/49ec856134ee424d8d5092495f937817` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.050828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:50.051287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.053395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.067423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 155/312] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:50.067972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] RBAC: Authorizing `identity:get_user(user_id=49ec856134ee424d8d5092495f937817)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.070917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.073576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:50.074737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 158/313] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:49 2022] GET /identity/v3/users/49ec856134ee424d8d5092495f937817 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:50.084796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:50.116992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.170562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.212663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330af92-7d21-46ff-8971-bfd46682d026 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.217821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1597e79-7ffb-4902-ae24-8a40a03d9334 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.256662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.257695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.258165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.258638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] PATH_INFO: `/v3/users/73971d7be9d449c19815fbd4c4d4a08e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.259966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] RBAC: Authorizing `identity:get_user(user_id=73971d7be9d449c19815fbd4c4d4a08e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.260314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.260964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.261243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.261518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.262252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, user.id=49ec856134ee424d8d5092495f937817, scope.system=all) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:50.262561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.262908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.263991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.265264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 156/314] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:50 2022] GET /identity/v3/users/73971d7be9d449c19815fbd4c4d4a08e => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:50.271563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 159/315] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 29 21:20:50 2022] GET /identity/v3/role_assignments?user.id=49ec856134ee424d8d5092495f937817&scope.system=all&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:50.271755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:50.387754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.395625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0151d65-65e1-474e-9f52-9a588665bfb9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.434946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.435635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.435913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.436207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.510502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.511961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.617777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.630377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 157/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:50.638032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:50.759287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af461d-168d-4605-b5f8-74bf466cf630 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.793942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:50.800342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.801252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.801730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.802190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.803535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.805331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83c3f9-90b8-4381-a424-5641d7b0b4b3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.807358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 160/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:50 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:50.813960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:50.922545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9179322-2347-4afc-afc2-44ab8b29990b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:50.961072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:50.961682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:50.961959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:50.962230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:50.962964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, scope.project.id=1a92d03657d140d5ad32882c7486d1af, user.id=73971d7be9d449c19815fbd4c4d4a08e) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:50.963266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:50.964731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5c38-71e5-45b5-bd41-b241a9a622f0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:50.971127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 158/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:20:50 2022] GET /identity/v3/role_assignments?user.id=73971d7be9d449c19815fbd4c4d4a08e&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:20:51.587455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d230f8-3e88-4bad-898a-2cc737a0644f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:51.587831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d230f8-3e88-4bad-898a-2cc737a0644f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:51.588033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d230f8-3e88-4bad-898a-2cc737a0644f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:51.589424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 161/319] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:51.595266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4d22c-9127-47d6-a860-197321069257 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:51.595884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4d22c-9127-47d6-a860-197321069257 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:51.595884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4d22c-9127-47d6-a860-197321069257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:51.616116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-13f4d22c-9127-47d6-a860-197321069257 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:51.617332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13f4d22c-9127-47d6-a860-197321069257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:51.676621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:51.689657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 159/320] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:51.955685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d230f8-3e88-4bad-898a-2cc737a0644f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:51.995857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a75-a864-49be-b937-84ac0d117f7c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:51.996167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a75-a864-49be-b937-84ac0d117f7c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:51.996405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a75-a864-49be-b937-84ac0d117f7c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:51.997800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 160/321] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:52.003670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.003948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.004265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.023439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:52.025006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:52.087688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d230f8-3e88-4bad-898a-2cc737a0644f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.107363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 161/322] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:52.131654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.132637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.133155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.133647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.146440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:52.151718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:52.152683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:52.153760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:52.157467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 162/323] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:52.161726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.278167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e746918-6463-4e0a-b420-0f9f467700b4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.304480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:52.325383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.326004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.326289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.326582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.327228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:52.327533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:52.329160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a9ad1-2fde-4f08-becd-91424af2cbb3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:52.344111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 162/324] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:52.345911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.357051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef427-22c1-4552-a56e-b9355fb2c994 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.357335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef427-22c1-4552-a56e-b9355fb2c994 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.357607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fef427-22c1-4552-a56e-b9355fb2c994 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.359432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 163/325] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:20:52.365040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.365336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.365599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.384314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:52.385466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:52.421495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:52.457821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 164/326] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:52.458725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.477818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9d05-3840-449e-bb16-f3d141778cff None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.525335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.526030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.526285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.526561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] PATH_INFO: `/v3/users/59c09c1851274b2398542ed8b4088227` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.527712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] RBAC: Authorizing `identity:get_user(user_id=59c09c1851274b2398542ed8b4088227)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:52.529241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:52.530883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 163/327] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/users/59c09c1851274b2398542ed8b4088227 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:52.538023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.590403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bd171-4b37-476d-939e-911b1c2c5a87 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.606036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:52.629573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.630314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.630594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.630868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.631981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:52.633447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:52.637120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 165/328] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:52.640654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.653149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e642aa-a0f1-43bb-ba7c-024c38a28552 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.681868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:52.690843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.691456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.691736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.692019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.692773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] RBAC: Adding query filter params (role.id=89fad9d5c83b4b04939f1ffc830b118d, user.id=59c09c1851274b2398542ed8b4088227, scope.system=all) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:52.693081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:52.694515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:52.708266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 164/329] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/role_assignments?user.id=59c09c1851274b2398542ed8b4088227&scope.system=all&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:52.730181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.758345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1e844-f36a-47cd-b6ad-4699034d45f6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.807720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.808351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.808735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.808999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] PATH_INFO: `/v3/groups` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.809656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:52.811529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:52.831733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 166/330] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 29 21:20:52 2022] POST /identity/v3/groups => generated 226 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:20:52.857942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aaac39-d984-43c9-97af-6e0063fa4869 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:52.906354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:52.907474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:52.907786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:52.908053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:52.921729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:52.927276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:52.928246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:52.929124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 165/331] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:20:52.934880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:52.981850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:53.043638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d19c3-9701-4969-b58d-296cc22805fa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:53.080505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:53.081164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:53.081441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:53.081717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:53.082385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:53.082683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:53.084060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:53.092728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 167/332] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:53.099447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:53.152443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:53.213498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c66e1-8c80-412b-9556-3ec1e4cab69e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:53.252867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:53.253570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:53.253851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:53.254125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] PATH_INFO: `/v3/users/49ec856134ee424d8d5092495f937817` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:53.255303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] RBAC: Authorizing `identity:get_user(user_id=49ec856134ee424d8d5092495f937817)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:53.256976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733df46-be1b-4fda-9efb-fd53869a29b4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:53.258825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 166/333] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:53 2022] GET /identity/v3/users/49ec856134ee424d8d5092495f937817 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:53.264489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:53.385764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337826-ab29-4821-b05b-a5a8c2ce6617 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:53.387422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:53.425602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:53.426268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:53.426550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:53.426913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] PATH_INFO: `/v3/system/users/49ec856134ee424d8d5092495f937817/roles/a5028fb57d8d4c1ea4f83d33bc1454d5` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:53.428352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49ec856134ee424d8d5092495f937817, role_id=a5028fb57d8d4c1ea4f83d33bc1454d5)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:53.429789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28c512-5364-4957-a75c-531638b2142b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:53.440559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 168/334] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 29 21:20:53 2022] PUT /identity/v3/system/users/49ec856134ee424d8d5092495f937817/roles/a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:20:55.334890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5702-94de-4ed7-baa5-4461c4855cfc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.335926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5702-94de-4ed7-baa5-4461c4855cfc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.336411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca5702-94de-4ed7-baa5-4461c4855cfc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.338371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 167/335] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:55.346000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe94b3-9f0f-427e-9228-519485bb54b5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.346925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe94b3-9f0f-427e-9228-519485bb54b5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.347409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe94b3-9f0f-427e-9228-519485bb54b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.369358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-66fe94b3-9f0f-427e-9228-519485bb54b5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:55.370903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-66fe94b3-9f0f-427e-9228-519485bb54b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:55.470217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 169/336] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:55.553257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3815e-2bff-4147-8515-04dfc28d0c3a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.554153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3815e-2bff-4147-8515-04dfc28d0c3a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.554885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3815e-2bff-4147-8515-04dfc28d0c3a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.557680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 168/337] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:55.563619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.564379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.565049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.590397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:55.591538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:55.607627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226aca-742a-4ec8-b0a8-5ea154fae890 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.608268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226aca-742a-4ec8-b0a8-5ea154fae890 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.608768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226aca-742a-4ec8-b0a8-5ea154fae890 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.610628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 169/338] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:55.616450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:55.617054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:55.617652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:55.647006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:55.648660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:55.687468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 170/339] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:55.748955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 170/340] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:20:56.057045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.086187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:56.112199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.219920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d8ec0-9874-45d9-a50e-ef5173fcbd79 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.221579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:56.278405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:56.279417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.279990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.280455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.294679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:56.298833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347360ca-b53b-4ce5-9729-9033aec932c9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-81ebdf6c-34c8-430b-90ad-f18d0890043e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:56.300208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:56.301160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:56.301965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:56.302566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 171/341] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:56.304419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1194-80d6-4b00-974d-9017468a722c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.309077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1194-80d6-4b00-974d-9017468a722c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.309526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e1194-80d6-4b00-974d-9017468a722c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.311485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 172/342] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:56.312470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e1194-80d6-4b00-974d-9017468a722c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.348402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:56.349218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.349587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.349950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.362553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b1b6c4ce-db11-45a9-a49a-4b3d86cb72af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:56.368350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:56.369363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:56.370175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 171/343] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:56.371799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.372177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.376649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.405978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:56.409132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:56.438135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:56.459905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e1194-80d6-4b00-974d-9017468a722c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.516491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:56.517458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.517944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.518386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.522534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 172/344] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:56.523475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.532994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:56.539043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:56.539981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:20:56.540842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:56.543686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 173/345] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:56.549094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.562229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:56.675166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a29238-e6e9-4095-953e-95abc83fd9d4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.738415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17eeb4-7ca1-43fd-a65d-78a5514c2b15 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.754216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:56.754839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.755115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.755384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.756056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:56.756352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:56.759898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:56.780148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 173/346] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:56.781030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.783865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:56.784791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:56.785296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:56.786080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:56.787010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:56.787507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:56.789124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:56.805419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 174/347] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:56.806608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:56.822097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:56.976365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d649e31-ed5b-4f78-9d78-28af69d51b84 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:56.991626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.000181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6e52-f058-44f3-a697-efcfe4533a6f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.019956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.020796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.021138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.021565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.048911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.050420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.050420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.050420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.050713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:57.050992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.052429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.065215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 174/348] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:57.065215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.124130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.125693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.141387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.218183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab8d70-fb79-4812-9753-7d72ec0c5bdf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.273790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.273790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.273790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.273790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.278077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 175/349] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:57.279948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.309983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.364165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.367443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.418313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faa7a7-e275-4efe-9db2-1ce6b5fa4e24 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.428781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.468196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.468892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.469209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.469418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.487101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7100a8ae-a57b-474c-ad7c-36fd517b69a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:57.492913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:57.493830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:20:57.494625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:57.496732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 176/350] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:57.498179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff3761-b820-4b29-b6fc-75340b8cd587 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.498654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff3761-b820-4b29-b6fc-75340b8cd587 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.499103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff3761-b820-4b29-b6fc-75340b8cd587 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.501616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 177/351] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:57.502548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff3761-b820-4b29-b6fc-75340b8cd587 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.515164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 175/352] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:57.516842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.604993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.652741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff3761-b820-4b29-b6fc-75340b8cd587 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.667489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de5ed2-819b-48eb-b53f-4f0e165a5f9c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.694580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.695806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.696094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.696364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.715798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.716575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.716889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.717504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.719541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.720360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.722343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 176/353] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:57.723011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.744408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.785602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.786639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.849459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65ff64-c93f-46bb-84d6-8eeccf8b3123 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:57.880021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:57.894000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:57.894622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.894900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.895167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.895840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:57.896136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:57.897604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4b1d3-4ad4-4320-9020-9274cc1a83c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:57.908390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 177/354] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:57.910030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fbadb-a9d6-47a6-9ef6-1709556a0c8a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:57.910312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fbadb-a9d6-47a6-9ef6-1709556a0c8a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:57.910578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fbadb-a9d6-47a6-9ef6-1709556a0c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:57.914446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 178/355] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:20:57.916202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:57.934961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6a0fbadb-a9d6-47a6-9ef6-1709556a0c8a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:57.936154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6a0fbadb-a9d6-47a6-9ef6-1709556a0c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:58.000143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.022757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 178/356] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:58.024931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee251d7-dfbe-45fe-a280-930302f07b30 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.025225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee251d7-dfbe-45fe-a280-930302f07b30 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.025501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee251d7-dfbe-45fe-a280-930302f07b30 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.027109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 179/357] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:58.027711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee251d7-dfbe-45fe-a280-930302f07b30 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.053983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6138-f227-428c-8c72-2e70ca7e4ecb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.102600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.103249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.103527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.104259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.105721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.107639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.109317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 179/358] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:57 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.110140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.166129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee251d7-dfbe-45fe-a280-930302f07b30 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.189281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.212415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.213089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.213361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.213628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.223126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.225959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.229245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 180/359] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.230587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.252245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5e97-ae47-47c3-9427-f7a5ffa97ca8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.296621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.297290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.297569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.297906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.300153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.301640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4aee06-0844-4d1c-b9c6-4fd3c3514d46 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.303294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 180/360] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.304490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.304711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.304985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.315526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.330758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:58.332068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:58.362886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ed8b2-f7cd-4c9a-a3be-b61df3b37f22 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.410317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.410317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.411035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.411035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] PATH_INFO: `/v3/users/49ec856134ee424d8d5092495f937817` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.412940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] RBAC: Authorizing `identity:get_user(user_id=49ec856134ee424d8d5092495f937817)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.414483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.414842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 181/361] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:20:58.415880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.416777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 181/362] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/users/49ec856134ee424d8d5092495f937817 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.417509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.484376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.550578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1dac8-0f26-4f7d-93a6-504fd3150c23 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8121a3-c977-40aa-ac26-244be5805b4b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.602465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.603121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.603392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.603664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.605389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.607222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.608569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 182/363] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.608961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.611718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.621303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.622145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.622578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.622983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.623859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:58.624162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.625640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.632133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 182/364] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=2be7d12a9a644f299290473830d5c537 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.633678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.753569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede683f-bb3a-4887-b8b6-afd1a974b59a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.788455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4ed55-db72-42b7-9941-2d62ab42ced0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.807486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.808147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.808428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.809252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.809547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.826337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:58.834053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:58.834994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:20:58.835838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 183/365] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:58.836512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.840010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:58.840506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:58.840736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:58.841045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:58.842680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:58.844462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:58.848694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 183/366] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:58.848694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:58.939838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:58.996136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668ff60-1582-4fb8-a189-55db31193c0c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:58.996972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1354-f8e8-4b8b-bf9e-d1b8d35c919d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.054213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.054213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.054213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.054577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.054822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.055088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:59.055432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.055802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.056155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.056433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.057450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.057554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] RBAC: Adding query filter params (role.id=a5028fb57d8d4c1ea4f83d33bc1454d5, user.id=49ec856134ee424d8d5092495f937817, scope.system=all) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:59.057626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.059287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.063759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 184/367] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:59.072971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.075599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 184/368] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 29 21:20:58 2022] GET /identity/v3/role_assignments?user.id=49ec856134ee424d8d5092495f937817&scope.system=all&role.id=a5028fb57d8d4c1ea4f83d33bc1454d5 => generated 510 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:59.082256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.161645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:59.245565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133ee9-fd85-47be-90b5-fe6715cf7173 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.293769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.294418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.294703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.294980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.311060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9ae9-7fa6-48e5-bf0d-02f4e580c7f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.311835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:20:59.318718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:20:59.319710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:20:59.320558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 185/369] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:20:59.321319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.350957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:59.386505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.387137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.387437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.387688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.388361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:59.388883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.390229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.408671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 185/370] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:59.409540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.487347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb5b84-7fcb-4331-870d-67fb43d314f9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.498660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:59.533331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.533677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea331767-0e0f-4cb1-b4bb-9fb1e758df29 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.534506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.535140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.535625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.537454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=1a92d03657d140d5ad32882c7486d1af) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:59.538005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.539706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.546338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 186/371] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:59.547860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.614698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.616058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.616567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.617081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.617994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:20:59.620466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.622163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.638039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 186/372] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:20:59.644747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:59.757512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:59.768450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1ee80-83be-48e8-b0b1-a7174f2d2f6f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.837156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.838212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.838832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.840462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:59.872477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452e4f5-2232-4997-b11a-27a1a993abbb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:59.903005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:59.947375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:59.949100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:59.975749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:59.976382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:59.976686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:59.976965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.113439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.115112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.175134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:00.240539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 187/373] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:00.248202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 187/374] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:00.248381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.254919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.403220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:00.411446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0303bc1-7a08-4962-9b17-415cdebde380 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:00.469848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26369ffe-d4bd-4696-9908-9ee977ba4e92 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:00.476086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:00.477008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.477581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.478059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] PATH_INFO: `/v3/projects/918734ce2c8c4362a4d1364fce951bb2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.483912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=918734ce2c8c4362a4d1364fce951bb2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.485685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.490046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 188/375] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/projects/918734ce2c8c4362a4d1364fce951bb2 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:00.493659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.527552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:00.544982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:00.545646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.545900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.546173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.548835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.550455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.552111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 188/376] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:00.557688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.639465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d378-8dd2-4965-9941-de6349ce9ba3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:00.677959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:00.678600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.678940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.679220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] PATH_INFO: `/v3/groups/2e09487da1444a76877e57b73875e66c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.685677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] RBAC: Authorizing `identity:get_group(group_id=2e09487da1444a76877e57b73875e66c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.687139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.689606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 189/377] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/groups/2e09487da1444a76877e57b73875e66c => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:00.695295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.790286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:00.805114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c1eb0-fb10-47a0-86b2-0ad5ad26bb11 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:00.839508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513e395-a114-483e-9de9-e2305dcbf82a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:00.880703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:00.881385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.881677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.881945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.882682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] RBAC: Adding query filter params (group.id=2e09487da1444a76877e57b73875e66c, role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=918734ce2c8c4362a4d1364fce951bb2) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:00.882998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.884445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.890541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 190/378] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/role_assignments?group.id=2e09487da1444a76877e57b73875e66c&scope.project.id=918734ce2c8c4362a4d1364fce951bb2&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:00.894886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:00.895566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.895845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.896121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.901528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:00.903071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcaa14-77db-4582-9a96-338acccdb863 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:00.905246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 189/379] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:00.912268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:00.985992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2f8c1-e6d6-4029-bcd2-da11d24dc1dc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.986324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2f8c1-e6d6-4029-bcd2-da11d24dc1dc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.986545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2f8c1-e6d6-4029-bcd2-da11d24dc1dc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:00.988111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 190/380] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:21:00.999220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472835-72a6-4dbc-a6a8-e398801b4439 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:00.999545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472835-72a6-4dbc-a6a8-e398801b4439 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:00.999763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472835-72a6-4dbc-a6a8-e398801b4439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.018860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:01.030610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-01472835-72a6-4dbc-a6a8-e398801b4439 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:01.032443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-01472835-72a6-4dbc-a6a8-e398801b4439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:01.116789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90828c-cd16-4012-806e-1adb17524eb1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:01.136024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 191/381] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:01.170267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:01.170892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:01.171177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:01.171457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.172335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=1a92d03657d140d5ad32882c7486d1af) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:01.172821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:01.174372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba123f46-9b16-431a-ac6c-f5aef59fa3a1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:01.180528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 191/382] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:00 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=2be7d12a9a644f299290473830d5c537 => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:01.414757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978591-7c62-4da1-9ff8-15d288bf7bc7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:01.415102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978591-7c62-4da1-9ff8-15d288bf7bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:01.415210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978591-7c62-4da1-9ff8-15d288bf7bc7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.416884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 192/383] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:01.422259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:01.422767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:01.423211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.434632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:01.446084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978591-7c62-4da1-9ff8-15d288bf7bc7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:01.452273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:01.459034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:01.604214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978591-7c62-4da1-9ff8-15d288bf7bc7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:01.610446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 192/384] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:01.665992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:01.667001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:01.667503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:01.667979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.673310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:01.685758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:01.692119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:01.693071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:01.693911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:01.696508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 193/385] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:01.705180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:01.848828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c87aa-6471-4245-ab90-923175cf6909 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:01.884696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:01.885504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:01.885781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:01.886051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:01.886714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:01.887011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:01.888367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:01.889792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:01.900623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 193/386] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:01.903017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.004013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.037440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f34bd-935a-4716-8fa6-18ebf4a28f2a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.100724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.101931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.102219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.102496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.103932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:02.106372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192bad2-c4ae-43cc-8b9e-057a120096e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:02.109306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 194/387] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:01 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:02.112067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9ef0-d2ca-4ea9-a538-902ce52eb188 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.112372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9ef0-d2ca-4ea9-a538-902ce52eb188 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.112815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a9ef0-d2ca-4ea9-a538-902ce52eb188 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.114466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 195/388] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:02.115443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a9ef0-d2ca-4ea9-a538-902ce52eb188 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.161849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde40ca-686f-4dc9-a10d-e87ceeede387 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.204637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:02.206369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.207493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.207919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.208331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.221814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d1eea8fa-b6dd-470c-a542-05bee668907d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:02.229247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:02.230345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:02.231268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:02.239152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 194/389] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:02.239152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.239152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.239152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.256669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a9ef0-d2ca-4ea9-a538-902ce52eb188 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.257806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:02.259238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:02.302498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.303361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.303793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.304221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.306008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:02.308497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:02.311052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 196/390] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:02.312528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.316763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:02.339409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 195/391] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:02.340450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.459606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fa4a1-6d5e-4973-a1f3-3ea6ebd27d9f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.507154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.507763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.508046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.508326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/groups/d2d4bea49b5942148cbae1abfecefb45/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.512003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, group_id=d2d4bea49b5942148cbae1abfecefb45, role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:02.513498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:02.528231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378934e4-3df0-49ef-a3d1-2e52a17c0e0d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.549551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 196/392] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 29 21:21:02 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/groups/d2d4bea49b5942148cbae1abfecefb45/roles/2be7d12a9a644f299290473830d5c537 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:02.571266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:02.579501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.580160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.580422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.580778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.581462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:02.581768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:02.583221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:02.593479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 197/393] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:02.605733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.688834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:02.738813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.768355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5094-4c66-4be0-abbd-be7b655fdc47 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.831439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.832457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.832954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.833407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.834937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:02.837495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:02.839932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 197/394] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:02.852095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.869487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9bc3a-a479-4f4f-9ed3-abc0d175f0e6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:02.914176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:02.914857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:02.915133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:02.915455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:02.930511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:02.938017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:02.938958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:02.939780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 198/395] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:02.945430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:02.985898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e91c5-f85b-4ec3-8813-38292ab42e18 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.001800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:03.026492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:03.029743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.030755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.032312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.032769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.034341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.036390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.038748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 198/396] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:03.044504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:03.077346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb718-0580-46ce-9ab3-ce7fffd8a4ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.114589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.115346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.115732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.116105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.116905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:03.117308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.119079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961757e-743e-45dd-ab31-8fa7066aec37 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.132783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 199/397] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:03.134337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c249b-0809-4db1-9b48-f18659c8a136 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.134747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c249b-0809-4db1-9b48-f18659c8a136 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.135125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c249b-0809-4db1-9b48-f18659c8a136 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.136742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 200/398] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:03.137958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c249b-0809-4db1-9b48-f18659c8a136 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:03.158118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa207f-7554-49bc-874b-7ee1d8c3b487 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.197584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.198544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.199020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.199491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/groups/d2d4bea49b5942148cbae1abfecefb45/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.202618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] RBAC: Authorizing `identity:create_grant(project_id=22afaf84b97c4e8b9e1b12d5385bdd77, group_id=d2d4bea49b5942148cbae1abfecefb45, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.205733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbab89-84b6-4967-a96a-59f0456a2c0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.229253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 199/399] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 29 21:21:03 2022] PUT /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77/groups/d2d4bea49b5942148cbae1abfecefb45/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:03.235565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.235565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.235565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.273489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:03.274759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:03.289401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c249b-0809-4db1-9b48-f18659c8a136 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.322949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:03.353921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:03.358243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.359313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.359844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.360545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.362276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.364066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.366476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 201/400] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:03 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:03.372938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:03.426078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 200/401] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:03.485245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31973d-4896-4979-b911-b1539ebd8399 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.522582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.523613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.524084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.524574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.525939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.527584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03227b-3070-4a8b-bbe9-33cf4ab75f43 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.529736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 202/402] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:03 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:03.537876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:03.538566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14eb3-1a75-4323-af6d-13e2abf13c9d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.538859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14eb3-1a75-4323-af6d-13e2abf13c9d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.539133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a14eb3-1a75-4323-af6d-13e2abf13c9d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.541765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 203/403] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:21:03.545655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.545939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.546218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.565290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:03.566509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:03.625180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:03.668647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 204/404] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:21:03.700432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a632eb2-82a1-4abb-8928-0f4f42c916e7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.748851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.749524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.749804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.750116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/groups/d2d4bea49b5942148cbae1abfecefb45/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.753916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, group_id=d2d4bea49b5942148cbae1abfecefb45, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:03.755370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:03.780248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 201/405] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 29 21:21:03 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/groups/d2d4bea49b5942148cbae1abfecefb45/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:03.812452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:03.939209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2c3de-232a-4ab3-a0ad-a480df0c2d30 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:03.976475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:03.977377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:03.977814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:03.978218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:03.990102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:03.995276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:03.996338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:03.997371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:03.999574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 205/406] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:04.009698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.014751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.052026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.088235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.153074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3612da-54c5-478a-83c7-7bb36fdf8f3f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.188193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b011bf-0c90-477b-aad0-82dedb6c3217 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.206346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.207449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.207933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.208394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.225045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.225601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.225883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.226235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.227271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:04.227780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.230334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.230596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:04.240274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 202/407] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:04.249634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:04.250627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:04.251526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:04.252337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:04.255133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 206/408] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:04.262797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.357403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.363132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fbef6-5978-4d24-baab-fb2f75151f0b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.409655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.410462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.410873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.411245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] PATH_INFO: `/v3/groups/2e09487da1444a76877e57b73875e66c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.412638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] RBAC: Authorizing `identity:get_group(group_id=2e09487da1444a76877e57b73875e66c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.414305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.417022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 203/409] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/groups/2e09487da1444a76877e57b73875e66c => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:04.422234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.506886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15d2b0-33d9-4d9d-89f1-95b6dc57d5b8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.508620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.564767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a810-4753-42f4-8c7d-f0030f7c30a2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.601162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.602092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.602475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.602855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] PATH_INFO: `/v3/projects/918734ce2c8c4362a4d1364fce951bb2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.604152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] RBAC: Authorizing `identity:get_project(project_id=918734ce2c8c4362a4d1364fce951bb2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.605790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.607956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 204/410] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/projects/918734ce2c8c4362a4d1364fce951bb2 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:04.614279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.617384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.618008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.618287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.618558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.619239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:04.619543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.620950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.641796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 207/411] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:04.647327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:04.669032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.725066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cc1f-077e-4550-a4ad-db1808da9b4b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.760012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.760754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.761045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.761329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] PATH_INFO: `/v3/projects/918734ce2c8c4362a4d1364fce951bb2/groups/2e09487da1444a76877e57b73875e66c/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.763246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] RBAC: Authorizing `identity:create_grant(project_id=918734ce2c8c4362a4d1364fce951bb2, group_id=2e09487da1444a76877e57b73875e66c, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.764932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.790979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 205/412] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 29 21:21:04 2022] PUT /identity/v3/projects/918734ce2c8c4362a4d1364fce951bb2/groups/2e09487da1444a76877e57b73875e66c/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:21:04.833019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555f28-5aca-4957-8796-aae8fc6e30da None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:04.873153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:04.873797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.874079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.874354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.876087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:04.877644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:04.879621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 208/413] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:04.884947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:05.013674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:05.024803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a51e8-6c03-447c-8081-23a30b6c5d54 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:05.067284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:05.067939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.068219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.068496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.069668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:05.071106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af4e2e-eee5-4f3f-bc55-b8e27b1d360d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:05.073064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 206/414] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:04 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:05.078498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:05.132792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:05.192348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd934-bb74-4ddf-a458-f3b699b6019a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.193040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd934-bb74-4ddf-a458-f3b699b6019a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.193512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dd934-bb74-4ddf-a458-f3b699b6019a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.196707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 207/415] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:05.202058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e52a-642b-45f3-b580-798b0498afd7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.202343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e52a-642b-45f3-b580-798b0498afd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.202618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e52a-642b-45f3-b580-798b0498afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.230663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828ff35-1bf9-4881-9f9b-ce44621f0e06 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:05.245582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a510e52a-642b-45f3-b580-798b0498afd7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:05.246793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a510e52a-642b-45f3-b580-798b0498afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:05.267337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:05.267996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.270389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.270671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af/groups/d2d4bea49b5942148cbae1abfecefb45/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.275929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a92d03657d140d5ad32882c7486d1af, group_id=d2d4bea49b5942148cbae1abfecefb45, role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:05.277404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4cb7-84ea-427f-8273-009cc3e0a814 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:05.308914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 209/416] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 29 21:21:05 2022] PUT /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af/groups/d2d4bea49b5942148cbae1abfecefb45/roles/2be7d12a9a644f299290473830d5c537 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:05.355276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 208/417] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:05.572421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24c7d-78c6-417c-9c11-b8d07ad16fae None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.572807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24c7d-78c6-417c-9c11-b8d07ad16fae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.573060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc24c7d-78c6-417c-9c11-b8d07ad16fae None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.574681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 210/418] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:05.581031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.581360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.581615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.601431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:05.602615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:05.682407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc24c7d-78c6-417c-9c11-b8d07ad16fae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:05.684683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 209/419] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:05.821831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:05.837627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc24c7d-78c6-417c-9c11-b8d07ad16fae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:05.879990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:05.880636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.880915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.881185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.899236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:05.909395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:05.910329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:05.911157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 211/420] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:05.916447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:05.945206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.060818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb70ed3-bdf3-48a1-b22d-b44d508a27a4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.080609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.110196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.110991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.111275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.111549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.112390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:06.112737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:06.114492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:06.124700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 210/421] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:06.130013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.248024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.263889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e1521-3f17-4349-98cd-04e78ae35fbb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.269402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318f884-faa9-4f03-a4da-e0c225cfe2cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.305346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.306239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.306663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.307073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.356001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.356668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.357147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.357549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.370735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0e352827-f390-4ab3-bc94-403e96b7d2b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:06.377453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:06.378378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:06.379189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:06.379820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:06.380093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7014ae-56e4-45fb-a044-faaff97259cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:06.382006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 212/422] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:06.383466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0ae3-2d33-4b7d-96fd-3cedff88b5c4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.383996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0ae3-2d33-4b7d-96fd-3cedff88b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.384466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad0ae3-2d33-4b7d-96fd-3cedff88b5c4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.385898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 213/423] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:06.391337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad0ae3-2d33-4b7d-96fd-3cedff88b5c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.408148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.469439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.490026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 211/424] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:06.493261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.493832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.494124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.512352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad0ae3-2d33-4b7d-96fd-3cedff88b5c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.517476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:06.518680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:06.550727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.551351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.551633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.551901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.552593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] RBAC: Adding query filter params (name=member) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:06.552911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:06.554309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:06.569094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 214/425] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:06.570003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.603653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 212/426] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:06.605217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.699277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.733114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0349-db20-4468-81ff-0a6b82463c23 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.753678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d58097-a500-41dc-811e-d054de389592 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:06.757153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:06.799016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.799671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.799951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.800225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.802776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:06.804253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:06.807329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 215/427] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:06.811345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:06.815589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:06.816217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:06.816496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:06.816794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:06.929792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:06.931236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:06.958439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156210fd-ff0b-4a63-8b68-38f70cc88003 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.002076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.002950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.003376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.003819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.005119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.006723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.008816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 216/428] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.009788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.032928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.060167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 213/429] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:07.060167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.072225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.131363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cca5a3-f116-4bc5-ae58-4beec97f081a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.186183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.187368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.187810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.188160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.193632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5f10d-73ec-4945-bed0-13f7962661f6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.206019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:07.213201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:07.214131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 217/430] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:07.215064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.238124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.238805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.239110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.239385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.240124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:07.240436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.241919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.248113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 214/431] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=2be7d12a9a644f299290473830d5c537 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.248912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.317192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.336624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.352992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26b6e6-469b-4ee2-98fa-90405c244840 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.366345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b922-928a-4400-9b4a-76ab1c9e7d49 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.391982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.393050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.393531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.394001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] PATH_INFO: `/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.395480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] RBAC: Authorizing `identity:get_project(project_id=22afaf84b97c4e8b9e1b12d5385bdd77)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.397184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278aa56-35ac-430f-a89d-0a9de6cbec67 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.399049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 218/432] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/projects/22afaf84b97c4e8b9e1b12d5385bdd77 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.400400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fad5a-26da-4876-93a6-24db0cd3caf3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.400901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fad5a-26da-4876-93a6-24db0cd3caf3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.401457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fad5a-26da-4876-93a6-24db0cd3caf3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.402886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 219/433] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:07.408907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fad5a-26da-4876-93a6-24db0cd3caf3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.412157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.413472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.413472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.413472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.413990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] RBAC: Adding query filter params (name=member) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:07.414303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.415658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5d462-a4d2-4510-a8a6-a8b8e81c473a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.425927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 215/434] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.426935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.427470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.427470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.449616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:07.450794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:07.536765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 216/435] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:07.538097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.546092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fad5a-26da-4876-93a6-24db0cd3caf3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.595095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.596126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.596625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.597117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.598976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.601173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.604196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 220/436] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.612796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.614458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.625554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.656109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd563b-5401-4cf4-ae12-9696fab01f77 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.692752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.693777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.694253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.694732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.741162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867803-0bb5-4a62-9e14-b95f6579e847 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:07.768610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.770073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.776919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:07.777525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.777812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.778084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.778811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:07.779120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:07.780538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb1c78-985d-4584-b160-561f62def3d3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:07.794451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 221/437] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=22afaf84b97c4e8b9e1b12d5385bdd77&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:07.794451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0a959-01b2-45ed-867b-dcb4a5410612 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.794451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0a959-01b2-45ed-867b-dcb4a5410612 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.794451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0a959-01b2-45ed-867b-dcb4a5410612 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.794451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 222/438] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:07.796128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.796411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.796717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.819015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:07.820447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:07.858508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.900241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 217/439] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:07.901055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:07.917431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:07.933642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 223/440] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:21:07.934629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.042575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebeaeb-0550-49e0-b91d-fb9064b5b95c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.078500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdaf468-de43-4df2-814f-91e19ebdb3e8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.092110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.093613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.093613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.096210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.112466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:08.119132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:08.120047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:08.120886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:08.121495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 218/441] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:08.122044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.122235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.124639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.124639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.124639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.125531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.127173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 224/442] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:07 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:08.129098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.132395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.165444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.223717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.260296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9f6e-28be-46e6-ae2b-c3ae39f43165 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.266134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cfd90-bb0f-49a0-9c1d-5ddf99dac56e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.305725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.306342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.306647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.306989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.307703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:08.308008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.309418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.310442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.311301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.311583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.311858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.313067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.314642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.316420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 225/443] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:08.319001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 219/444] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:08.323313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.323439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.426419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.439664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81ce7b-1e0d-422b-9f7d-ac86f9739cef None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.441886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b83c2e-0455-4b9c-bdee-916cecff134a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.474513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.475117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.475394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.475659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.476382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=89fad9d5c83b4b04939f1ffc830b118d, scope.project.id=1a92d03657d140d5ad32882c7486d1af) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:08.476725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.476825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.477415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.477690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.477958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.478186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.480764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.484200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 220/445] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=89fad9d5c83b4b04939f1ffc830b118d => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:08.485046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.490627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:08.495628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:08.496773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:08.497612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 226/446] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:08.502968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.617968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82ef61-fcad-4be2-8103-199dd84fab2c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.619450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba065-2eb0-4d48-a7d0-427895f661df None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.660932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.661815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.662244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.662681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.665657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.666268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.666561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.667637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.668315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:08.668657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.670039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.682864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 227/447] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:08.685919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.693947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.750605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.752173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.760666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.816901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55492d5b-2ac7-4a0a-9339-6dee48b53caa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.855364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.855978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.856261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.856536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.869341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 221/448] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:08.874743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.924994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.926468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.926740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.974542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207606d-cf88-4ec1-9b0f-47768c4891e9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.984123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:09.010439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.011070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.011348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.011620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] PATH_INFO: `/v3/projects/918734ce2c8c4362a4d1364fce951bb2` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.013124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] RBAC: Authorizing `identity:get_project(project_id=918734ce2c8c4362a4d1364fce951bb2)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.014587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.017094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 222/449] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/projects/918734ce2c8c4362a4d1364fce951bb2 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:09.022481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:09.027956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 228/450] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:09.034008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:09.120679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02822d68-3da3-4bf7-83c3-9fc32f43bf82 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:09.135530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ab155-0704-4b10-a369-39851f87f773 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:09.139989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:09.154964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.155585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.155861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.156260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] PATH_INFO: `/v3/groups/2e09487da1444a76877e57b73875e66c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.158239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] RBAC: Authorizing `identity:get_group(group_id=2e09487da1444a76877e57b73875e66c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.159650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.161264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 223/451] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:09 2022] GET /identity/v3/groups/2e09487da1444a76877e57b73875e66c => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:09.166968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:09.172789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.173409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.173687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.173957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] PATH_INFO: `/v3/projects/1a92d03657d140d5ad32882c7486d1af` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.175080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] RBAC: Authorizing `identity:get_project(project_id=1a92d03657d140d5ad32882c7486d1af)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.176520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.178539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 229/452] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:21:09 2022] GET /identity/v3/projects/1a92d03657d140d5ad32882c7486d1af => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:09.183721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:09.210941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:09.267469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad14f-d988-4897-8bdf-d6128c32b362 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:09.283587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564c20-d497-4529-a7bf-504ec4c4daf6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:09.302908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.303513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.303792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.304066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.304812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] RBAC: Adding query filter params (group.id=2e09487da1444a76877e57b73875e66c, role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=918734ce2c8c4362a4d1364fce951bb2) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:09.305119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.306535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.312319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 224/453] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:09 2022] GET /identity/v3/role_assignments?group.id=2e09487da1444a76877e57b73875e66c&scope.project.id=918734ce2c8c4362a4d1364fce951bb2&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:09.318603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.319248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.319521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.319791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] PATH_INFO: `/v3/groups/d2d4bea49b5942148cbae1abfecefb45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.321014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] RBAC: Authorizing `identity:get_group(group_id=d2d4bea49b5942148cbae1abfecefb45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.322507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0d9f0-56e4-419f-b2b7-7189257d39ba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.324153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 230/454] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 29 21:21:09 2022] GET /identity/v3/groups/d2d4bea49b5942148cbae1abfecefb45 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:09.330966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:09.389995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:09.443242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be9f89-21ad-4c31-95a8-bc74be6af682 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:09.485923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:09.486549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:09.486830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:09.487104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:09.487839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] RBAC: Adding query filter params (group.id=d2d4bea49b5942148cbae1abfecefb45, role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=1a92d03657d140d5ad32882c7486d1af) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:09.488148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:09.489622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df021a-51c5-4590-b40a-9c6dc1480289 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:09.496426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 225/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 29 21:21:09 2022] GET /identity/v3/role_assignments?group.id=d2d4bea49b5942148cbae1abfecefb45&scope.project.id=1a92d03657d140d5ad32882c7486d1af&role.id=2be7d12a9a644f299290473830d5c537 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:12.217773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9cd1-5f3f-4f8b-bc4d-45a0cb47647b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.218267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9cd1-5f3f-4f8b-bc4d-45a0cb47647b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.218267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9cd1-5f3f-4f8b-bc4d-45a0cb47647b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.220700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 231/456] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:12.229311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1864-f191-45c0-bc0f-1f82dc47763f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.229311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1864-f191-45c0-bc0f-1f82dc47763f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.229311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec1864-f191-45c0-bc0f-1f82dc47763f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.258893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-caec1864-f191-45c0-bc0f-1f82dc47763f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:12.260415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-caec1864-f191-45c0-bc0f-1f82dc47763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:12.366679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 226/457] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:12.385302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00dd9f-ae0c-4d9d-afe3-6bc7ac190c07 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.385608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00dd9f-ae0c-4d9d-afe3-6bc7ac190c07 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.385848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00dd9f-ae0c-4d9d-afe3-6bc7ac190c07 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.387447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 232/458] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:12.393470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9e7e-7ab1-47c7-a15b-d34ba0335f64 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.394023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9e7e-7ab1-47c7-a15b-d34ba0335f64 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.394520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc9e7e-7ab1-47c7-a15b-d34ba0335f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.420506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-00cc9e7e-7ab1-47c7-a15b-d34ba0335f64 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:12.422661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-00cc9e7e-7ab1-47c7-a15b-d34ba0335f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:12.480409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:12.515266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 227/459] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:12.739154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3c12-53fb-4ec6-b7eb-b66352f7b0bc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.739525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3c12-53fb-4ec6-b7eb-b66352f7b0bc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.739675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f3c12-53fb-4ec6-b7eb-b66352f7b0bc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.740982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 233/460] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:12.746662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:12.746946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:12.747205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:12.766924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f3c12-53fb-4ec6-b7eb-b66352f7b0bc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:12.774159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:12.775553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:12.933673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 228/461] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:12.934731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:12.950036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:12.959392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f3c12-53fb-4ec6-b7eb-b66352f7b0bc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:12.999048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.000715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.001200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.001683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.013218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.015096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a553c094-f6e9-42d7-8172-897a11e393d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:13.031631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:13.032929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:13.034032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:13.034032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.034032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:13.040359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 234/462] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:13.042124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd924c89-e3bb-4499-afb7-274ac03f9376 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.042582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd924c89-e3bb-4499-afb7-274ac03f9376 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.042995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd924c89-e3bb-4499-afb7-274ac03f9376 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.044762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 235/463] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:13.046182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd924c89-e3bb-4499-afb7-274ac03f9376 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.080088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38642b8-2410-4bff-be2f-3a04570d668b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.089253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.124135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.125514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.126106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.126532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.139781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.142001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-19b89f98-754a-4530-bcc0-81015a3ad249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:13.161240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:13.162242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:13.163479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:13.163479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.163479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:13.164706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd924c89-e3bb-4499-afb7-274ac03f9376 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.165536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 229/464] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:13.167221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.167668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.168085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.190576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:13.192428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:13.207599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.208213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.208491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.208794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.209460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:13.209762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.211376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.236649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 236/465] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:13.238235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.250274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.282770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 230/466] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:13.284338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.383771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232968-d25d-4c26-b010-97dae617f25d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.394837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.411734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1e094-e279-49d6-9422-33fd1746a9e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.424891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.425873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.426019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.426386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.427144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:13.427574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.429131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.442027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 237/467] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:13.447411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.454659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.455559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.455998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.456414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.457246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.458993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.487895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 231/468] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:13 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:13.489325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.586300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.649920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b375-e7a1-408b-8753-0f85e7460758 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.673539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d0c54-8435-44e0-8c79-64e494c48909 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.713518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.714680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.715123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.715589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.724977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.725968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.728212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.728212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.728212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.730602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3d5cd-3d53-4c35-80ae-984dfeab2cfc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.738792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.740288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.759045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:13.763664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:13.765040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:13.765964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:13.765964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:13.765964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:13.765964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 238/469] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:13.765964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.777008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 232/470] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:13 2022] POST /identity/v3/users => generated 246 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:13.779581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a5a9a-4842-47a4-b225-c1046f31cbda None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.780067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a5a9a-4842-47a4-b225-c1046f31cbda None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.780554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a5a9a-4842-47a4-b225-c1046f31cbda None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.783536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 233/471] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:13.785742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a5a9a-4842-47a4-b225-c1046f31cbda None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.924954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a5a9a-4842-47a4-b225-c1046f31cbda None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.967398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf41357-eedc-4fb1-9ed0-446e60a53afd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:13.972814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:13.974167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.974454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.974732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.975401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:13.976304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:13.978168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498be3b9-aa6e-4177-a73a-9a60fa53c79e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:13.991729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.996398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 234/472] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:13.997795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.998079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.998353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.019717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:14.021034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:14.022777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.023705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.023994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.024268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.042022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.043516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:14.062775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:14.063913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:14.064999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:14.064999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:14.064999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:14.064999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 239/473] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:14.065524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:14.096779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:14.112259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 235/474] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:14.113422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:14.257620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bf89f-0d25-4be6-9d8a-d6240c090d3e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:14.311341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801fa81-1619-4aca-bb73-657da9e31e9e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:14.332658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.333609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.334085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.334551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.335418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.337106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.375510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.378910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.383837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 240/475] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:14 2022] POST /identity/v3/users => generated 248 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:14.384186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.384661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.385665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:14.386166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.387838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.402600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 236/476] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:14.409124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:14.445632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:14.575521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e474e3-6f04-46bc-89f8-ba8d4566215b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:14.631618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.632154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.632439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.632768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.633435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.634895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.660119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 241/477] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:14 2022] POST /identity/v3/users => generated 249 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:21:14.689944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:14.749689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:14.809586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06cabf-2e09-4fc4-9576-ab1af035525d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:14.864206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.865286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.865753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.866228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.867910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.870038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.885463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:21:14.885463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 21:21:14.885463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [parameters: {'id': '2775b26978414b64b2c040e314cf8ff6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 21:21:14.885463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 29 21:21:14.916849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:21:14.917787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:21:14.918718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:21:14.920479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:21:14.923333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:21:14.925387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '2775b26978414b64b2c040e314cf8ff6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:14.933133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 21:21:14.935118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:14.937091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 237/478] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 29 21:21:14 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:14.937758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:15.025381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:15.044984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f44c-9ce4-4779-ba81-10f1659cf056 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:15.082424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:15.083057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:15.083337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:15.083609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:15.094912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-82858f0e-25f8-494e-aa0e-19a6c0ab4249 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:15.100285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:15.102466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:15.104382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 242/479] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:15.110451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:15.322384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302604b9-58b5-4685-92d4-aa20092e2073 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:15.353995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:15.390907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:15.392249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:15.392748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:15.393227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:15.394473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:15.395009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:15.397613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1062c98-5828-4f13-bf3f-f7669244ee4f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:15.413022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 238/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:16.136690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d02ad-a6d8-4f1e-ae62-8b6b23a241ce None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.137172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d02ad-a6d8-4f1e-ae62-8b6b23a241ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.137172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d02ad-a6d8-4f1e-ae62-8b6b23a241ce None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.138751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 243/481] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:16.144695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2c8-e1e3-4e04-bc82-c5c21a0ace5e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.145041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2c8-e1e3-4e04-bc82-c5c21a0ace5e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.145435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622e2c8-e1e3-4e04-bc82-c5c21a0ace5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.164515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2622e2c8-e1e3-4e04-bc82-c5c21a0ace5e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:16.165742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2622e2c8-e1e3-4e04-bc82-c5c21a0ace5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:16.262842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 239/482] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:16.691391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d02ad-a6d8-4f1e-ae62-8b6b23a241ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:16.779044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ebe6c-37bd-450e-b3d3-ee34db705973 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.779044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ebe6c-37bd-450e-b3d3-ee34db705973 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.779044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ebe6c-37bd-450e-b3d3-ee34db705973 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.780118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 240/483] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:16.789896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0505-6a04-4b06-b79d-6a4d407e9135 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.790315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0505-6a04-4b06-b79d-6a4d407e9135 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.790689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b0505-6a04-4b06-b79d-6a4d407e9135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.833657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-966b0505-6a04-4b06-b79d-6a4d407e9135 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:16.835033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-966b0505-6a04-4b06-b79d-6a4d407e9135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:16.838997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d02ad-a6d8-4f1e-ae62-8b6b23a241ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:16.911933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:16.913225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.913931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.914618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.942448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3560a2c0-c6ad-4c07-acbd-3d0094f1e9a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:16.948435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:16.949412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:16.950234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 244/484] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:16.950916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72943077-58cb-4730-a2a9-dd3ab4c80048 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.956948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72943077-58cb-4730-a2a9-dd3ab4c80048 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.956948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72943077-58cb-4730-a2a9-dd3ab4c80048 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.958700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 245/485] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:16.959599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72943077-58cb-4730-a2a9-dd3ab4c80048 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:16.962994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:16.983111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 241/486] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:16.993540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc270a-1734-4608-91be-02efbb3b7874 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.994809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc270a-1734-4608-91be-02efbb3b7874 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.995081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc270a-1734-4608-91be-02efbb3b7874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.037737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9ebc270a-1734-4608-91be-02efbb3b7874 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:17.038954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9ebc270a-1734-4608-91be-02efbb3b7874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:17.101268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.176382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72943077-58cb-4730-a2a9-dd3ab4c80048 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.184455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 242/487] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:17.206005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7078fb9-f3f2-4d4c-a362-cdfb010d2ce5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.206191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7078fb9-f3f2-4d4c-a362-cdfb010d2ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.206461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7078fb9-f3f2-4d4c-a362-cdfb010d2ce5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.208519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 243/488] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:17.218233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.219335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.220048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.242665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.243284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.243558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.243821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.244493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:17.248130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:17.249627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:17.260859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:17.263791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:17.268827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 246/489] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:17.278763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.351653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.396628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 244/490] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:17.398447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.400972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916e782-02c1-458b-bbf6-3cb471d5191b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.442826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.444108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.444633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.445126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.458829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:17.461075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:17.478559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:17.479652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:17.480533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:17.480533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:17.480533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:17.482421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 247/491] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:17.489094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.519476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.527656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc714a-a4f5-4dc4-9b65-619f4cbde005 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.581296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4140-cf96-4472-8112-66400168143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.581961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4140-cf96-4472-8112-66400168143e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.582561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4140-cf96-4472-8112-66400168143e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.583111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f4140-cf96-4472-8112-66400168143e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.596253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f4140-cf96-4472-8112-66400168143e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-353f4140-cf96-4472-8112-66400168143e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:17.601582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:17.602494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:17.603794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 245/492] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:17.604751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4140-cf96-4472-8112-66400168143e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.630199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3456d-6609-4ae2-a49c-4f6104d6ab87 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.654564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.680387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.681624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.682051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.682599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.683432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:17.684112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:17.686409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:17.711417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 248/493] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:17.712550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.740376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f4140-cf96-4472-8112-66400168143e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.786522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.787157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.787415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.787690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.802278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:17.807993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:17.809115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 246/494] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:17.810097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.833736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.838596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09cef-f664-4c6e-a5c7-cc59bdfbc450 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.886483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.887439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.887941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.888486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.889161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:17.889580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:17.891337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:17.905009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 249/495] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:17.905694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:17.943659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602320e-5346-4e83-bcb0-75beb6500615 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.960887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.989228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.989795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.990043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.990311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.000544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.002771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:18.025843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:18.026829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:18.027905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 247/496] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:18.028891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.043481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3490e-3835-448c-9fa2-9a50d92ce49e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.083812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.084493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.084815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.085094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.097853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:18.103507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:18.104404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:18.105272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 250/497] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:18.105985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.120905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.146564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7a948-f8b3-4de6-860f-5819a7174d55 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.182475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.183112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.183373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.183651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.184319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:18.184647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.186017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.193958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 248/498] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:18.194765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.213815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.218172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69af13-d254-40ef-bae8-7c146e26583c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.258943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.259486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.259767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.260046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.271671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.273190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:18.288166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:18.289107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:18.290181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:18.290181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.290181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:18.290181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 251/499] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:18.290181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.301109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bfcec-9f96-4d26-a764-13dea4764d29 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.334431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.335026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.335304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.335571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.336288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:18.336616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.338046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbaa5e4-fcee-405c-976c-ced0309da041 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.356038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.376642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 249/500] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:18.377891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3df-6200-42aa-92d6-d0bf7e0754ac None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.378181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3df-6200-42aa-92d6-d0bf7e0754ac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.378458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18b3df-6200-42aa-92d6-d0bf7e0754ac None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.383800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 250/501] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:18.383800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b3df-6200-42aa-92d6-d0bf7e0754ac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.392367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097489ba-a4ab-49e7-9a32-9e24a5b497ae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.426064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.426661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.426941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.427215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.427866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:18.428174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.429585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.437751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 252/502] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:18.438596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.452848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.485818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18b3df-6200-42aa-92d6-d0bf7e0754ac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.520650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.521208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.521482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.521750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.532278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.533809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.538267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0f51f-b942-450d-b73a-7ecfc42bdbd3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:18.551831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:18.552825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:18.553692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:18.553692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.553692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:18.553692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 251/503] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:18.553927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.574218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.574821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.575103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.575377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.576018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:18.576321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.578373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69a4ef-3fa7-498f-817b-e876dfd10729 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.591348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 253/504] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:18.592518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.592826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.593105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.605302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.611648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:18.612889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:18.657518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0520a5-fd5d-48fa-9544-15f6fb774ea3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.681735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 254/505] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:18.682391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.684486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.699959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.700796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.701216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.701629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.784248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c9c2c-bb3d-4479-904e-923106633c8e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:18.784851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.786323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.821803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:18.822431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:18.822709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:18.822984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:18.834050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:18.835146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.835576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:18.852060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:18.853053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:18.853919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:18.853919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:18.853919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:18.853919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 255/506] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:18.853919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.904808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 252/507] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:18.905529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:18.921716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:18.969299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34d38f-04bd-47b4-be7c-239f836a5282 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.003376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.003984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.004272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.004534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.005204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.005502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.006877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.008360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9604-b855-4da1-b872-65a1feb4af01 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.019252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 256/508] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.020048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.043183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.043759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.044032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.044298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.051481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.052965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:19.064620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:19.065559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 253/509] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:19.066515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.076989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.125060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9bdb-bf96-477e-a58c-4dfd0a6a2a9b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.139027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.160310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.160943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.161222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.161490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.162123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.162422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.163787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.165340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e341e-d869-402d-8095-8c2ebfea59cb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.175165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 257/510] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.175788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.200894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.201523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.201805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.202081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.213419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.214880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:19.229656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:19.230776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:19.231666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:19.231666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:19.231666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:19.231666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 254/511] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:19.231960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.279861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd067b-2754-4acd-baaf-a0afd3db8464 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.291163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.313509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.314044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.314320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.314588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.325648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.327088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.333817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d648e-120e-409f-bb6c-59a66fee9c21 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:19.341575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:19.342617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:19.343531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:19.343531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:19.343531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:19.343531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 258/512] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:19.343531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.368179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.368842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.369128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.369396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.372446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.377949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.379405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:19.391153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:19.392150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 255/513] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:19.393178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.446602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b7386-42ea-4611-9be5-a03429aaa9fe None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.480548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.481168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.481441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.481711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.482428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.482727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.484166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.491939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d01df-7cbd-4286-9a5b-f6f702456f8f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.512007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.520227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 259/514] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.521170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.528057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.528725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.529010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.529282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.537524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.539112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:19.550871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:19.551841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 256/515] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:19.552818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.610991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.627480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e56-ffc1-4d1f-abdf-c790d7c5a7e7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.654028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de799d-b59a-4a8a-a544-7238cd487004 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.661881 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.662490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.662772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.663051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.663696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.664003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.665447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.677411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 260/516] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.678202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.687952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.688566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.688884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.689165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.689809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.690113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.691502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.703416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 257/517] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.704232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.746875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.790082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa14976-dd73-49b2-85cc-2ad8ef98000b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.808263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86215f95-3d78-41c1-91e3-29e1ee00db2f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.828143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.828775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.829480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.829771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.830507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.830810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.832283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.843149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.843750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.844024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.844360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.849487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 261/518] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:19.850310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:19.852948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.917515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.918964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:19.952655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879133b-01d0-4b16-b1e2-698828d6c0f3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:19.958843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.987126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:19.987718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.987998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.988263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.989015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:19.989312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:19.990737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.009490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 262/519] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:20.010244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.020941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 258/520] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:20.021755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.068856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.110131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0ec00-3d48-4cc7-9241-5f120fc22510 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.121525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd962594-46be-4856-8117-e0c8dffbc212 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.145452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.146077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.146353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.146623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.153192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.154650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.155319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.155954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.156236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.156505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.157437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.158873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:20.167104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:20.167968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:20.168875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 263/521] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:20.169359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.174439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.186580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 259/522] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:20 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:20.187949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.288658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf231f-af70-4b42-ae58-99f05c04dd58 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.305427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6d5be-65ce-4765-9588-52fc70d3db91 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.313031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.330518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.331226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.331511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.331783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.348721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.349347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.349679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.349960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.412084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.413993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.422680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.429781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.431225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.525600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 264/523] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:20.525600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.541004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 260/524] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:20.542164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.553077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.641535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbda7a8-23e2-4379-87fd-0a5b599cb25d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.646783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.654239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efeaea-9a11-4cc5-a666-d86563b5e1aa None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.681570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.682427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.682848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.683118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.692751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.693358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.693638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.693913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.694599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:20.694803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.694915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.696265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.696361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.714722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 261/525] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:20.715586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:20.716036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:20.717021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:20.718801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:20.718801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.718801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:20.721306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 265/526] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:20.722754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.804979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.831135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4c98e-e434-4459-9c28-20c719a28c16 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.838152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda1344-4faa-4865-98c0-646913310d07 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.870664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.871296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.871595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.871831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.878366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.878995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.879272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.879542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.885325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.887184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.890225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.891770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:20.903474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:20.905292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:20.907001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:20.907001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.907001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:20.907001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 262/527] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:20.907001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.917956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:20.918963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:20.919979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:20.920923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:20.920923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:20.920923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:20.920923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 266/528] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:20.922700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.016883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529bcbb-8c73-4a08-8bb0-2e7206fe9a21 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.056090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.056811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.057119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.057392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.058128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0abda3580f52421d816eac7bee95b492) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.058239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80a135-9517-47ab-bd35-dee4c3eb1ac1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.058440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.059880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.076911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 263/529] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/role_assignments?user.id=0abda3580f52421d816eac7bee95b492&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.076911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.101460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.113004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.113832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.114179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.114673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.115335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.115684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.117232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.134819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 267/530] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.135965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.168464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.208636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf762-7c28-48a2-86ac-7c87c20d7bb5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.260455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.261381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.261804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.262241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.263471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.263943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.266668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.278263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c22f-0f82-4f8d-84a7-1b2c7b0adeee None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.289212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 264/531] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.290439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.324771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.325422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.325767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.326359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.328208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.328733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.330602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.348478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 268/532] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.349962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.413209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.423526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6fe44-0a66-4d0e-aea3-1e32c0d17f2e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.459100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.466715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.467469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.468087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.469051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.472476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d66398-0abc-4152-a72d-243a35fbb64e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.480326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.481932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:21.500547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:21.501720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:21.502592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 265/533] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:21.503085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.518446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.519416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.519931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.520402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.527933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.529800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:21.542588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:21.543497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.544349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:21.548622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 269/534] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:21.549577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.619693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c562913-7942-498c-972a-3729322ebc32 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.658169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.658784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.659112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.659385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.661146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c75653-d114-4309-9706-36e8f5a68594 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.663711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.667956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.669518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:21.682946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:21.684026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:21.685274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:21.690105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 266/535] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:21.691054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.704976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.706274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.706985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.707442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.708297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.708833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.710667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.731659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 270/536] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.733007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.750464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.805199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc84b-c7c7-46a5-a226-79645c33fbf4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.840917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.841523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.841798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.842067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.842751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.843050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.844960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ffbb3-a683-4148-a51c-33d07c8ecfad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:21.845226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.863405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 267/537] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.864311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.884210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:21.885239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:21.885722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:21.886194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:21.887117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:21.887623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:21.889313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:21.903468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 271/538] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:21.904927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:21.934006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:21.989413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294de99a-1df1-4159-95e3-e3559beeb435 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:22.007847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:22.023887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33286d99-972b-46b4-9168-f0f3fac01613 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:22.036968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:22.037857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.038640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.039055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.040234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:22.040716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:22.042580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:22.050685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 268/539] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:22.052156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:22.085558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:22.086284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.086666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.087045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.088034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=00f82e19c26b45d2a4ac373665f8c9d5) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:22.088366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:22.089947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633d5ef-044b-426b-b981-f29ab9fdb4ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:22.105658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 272/540] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:21 2022] GET /identity/v3/role_assignments?user.id=00f82e19c26b45d2a4ac373665f8c9d5&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:22.182081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b765f-8cf9-43f0-aab5-dedce6950988 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:22.225148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:22.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.226475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.226925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.227839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=a7af5dafacef4215b39c06af7c66c77c) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:22.228299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:22.228570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d308-b1b3-4e5c-a3b1-e741ceadf771 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.228910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d308-b1b3-4e5c-a3b1-e741ceadf771 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.229195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d308-b1b3-4e5c-a3b1-e741ceadf771 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.230561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:22.230900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 273/541] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:22.237571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.237854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.238132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.239726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 269/542] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:22 2022] GET /identity/v3/role_assignments?user.id=a7af5dafacef4215b39c06af7c66c77c&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:22.273940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:22.275580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:22.367170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 274/543] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:22.694163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:22.759926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:22.842720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96af05-3fd9-486c-ab53-bdb9d221001c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:22.911146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:22.912176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:22.912639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:22.912927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:22.937710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:22.943389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:22.944344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:22.945183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:22.946257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 270/544] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:22.950909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:22.976841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:23.091096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c351618-f1c2-434d-9eb7-8de6ef33824f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:23.141870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:23.142951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.143441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.143912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.145190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:23.145742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:23.148362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:23.161661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 275/545] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:23.169644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:23.317198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:23.320222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130deb8-513b-4168-9a6f-dcfd3993ad05 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:23.366363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:23.367065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.367350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.367622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.384806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:23.386285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8fc06162-f1de-4258-958f-6f6a874f4cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:23.401391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:23.402500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:23.403569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:23.403569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:23.403569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:23.403569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 271/546] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:23.409635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:23.486567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d0db-8ab4-4a54-b4c9-a1ecbcf42377 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.486897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d0db-8ab4-4a54-b4c9-a1ecbcf42377 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.487132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d0db-8ab4-4a54-b4c9-a1ecbcf42377 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.492624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 272/547] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:23.493810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.494271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.494378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.521669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:23.523726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:23.573029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:23.612095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9d63-1559-49b0-b446-bf713aa3796d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:23.663275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:23.663975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.664292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.664664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.665336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:23.665738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:23.667189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:23.667500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 273/548] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:23.682097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 276/549] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:23.688401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:23.800616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:23.818175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba5349-9142-4231-b5b8-65a4a7a64ac3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:23.868195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:23.869087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.869543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.869963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.877392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:23.879054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:23.892067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:23.893338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:23.894278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:23.899991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 274/550] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:23.902524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.030782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230b6cf-528a-4d75-94e7-da2e59b45151 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.052062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.072943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.073864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.074289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.074717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.075586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:24.076022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.078256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.088568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.104408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 277/551] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:24.105880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.211731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb3fbc-bb21-4187-ba1d-15afd20a664e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.220872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebf7c6-ee9a-47aa-b0b1-e363c83301ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.259153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.260616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.261078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.261500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.305844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.306646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.307026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.307400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.339477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.339851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:24.341555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.344242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:24.346318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:24.347362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:24.348205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:24.354603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 275/552] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:24.360344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.363553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.465011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 278/553] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:21:24.471660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.585026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.588298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff731f-92e1-4535-b361-9128af3dc962 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.623181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1328d9-faf4-4980-b855-7df78a5c3b4c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.632761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.634103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.634560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.635436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.651022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.653763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.660646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.661346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.661697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.661982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.662653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:24.662961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.664448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f1aa56-088e-432b-b6ba-9a12188c9219 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:24.672762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:24.673734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:24.674621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:24.674621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:24.674621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:24.674805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 276/554] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:24.674805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f32e8-f479-4ae8-9e15-53afd933a03f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.674805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f32e8-f479-4ae8-9e15-53afd933a03f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.675512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f32e8-f479-4ae8-9e15-53afd933a03f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.677250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 277/556] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:24.678116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 279/556] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:24.678474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cfcf3-e0dc-4322-8718-aa9ff6c87c98 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.678758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cfcf3-e0dc-4322-8718-aa9ff6c87c98 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.679033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cfcf3-e0dc-4322-8718-aa9ff6c87c98 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.679799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.680537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 278/557] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:24.681268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cfcf3-e0dc-4322-8718-aa9ff6c87c98 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.750829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.797245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cfcf3-e0dc-4322-8718-aa9ff6c87c98 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.802214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13151-c0ba-4c72-bea1-fc141aa26689 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:24.832070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.832716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.832995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.833271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.833910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:24.834213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.835594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b1b74-c0c3-4ebf-8518-c55f50f9326f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.835693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:24.836223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.836502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.836806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.847467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:24.847832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 279/558] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:24.848943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:24.849129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1ac1-5530-4d41-be1e-40ae42a8bb15 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.849407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1ac1-5530-4d41-be1e-40ae42a8bb15 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.849682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c1ac1-5530-4d41-be1e-40ae42a8bb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e6b9b3c1-e927-46db-ac15-7e19f89f1ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:24.868405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:24.869399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:24.875862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:24.875862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:24.875862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:24.875862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 280/559] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:24.876142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-447c1ac1-5530-4d41-be1e-40ae42a8bb15 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:24.876243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.876243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.876401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-447c1ac1-5530-4d41-be1e-40ae42a8bb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:24.876496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.879006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:24.891005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:24.892195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:24.941656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 280/560] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:24.942903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2d9d-1dbd-4236-b405-ffbec4c6f1d7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:24.943186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2d9d-1dbd-4236-b405-ffbec4c6f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:24.943464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2d9d-1dbd-4236-b405-ffbec4c6f1d7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:24.944726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 281/561] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:24.945410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2d9d-1dbd-4236-b405-ffbec4c6f1d7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:24.962327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 281/562] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:24.963407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.034833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.078590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cf2cf-c7f1-4966-bd2f-5dd7bf14be3f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.100852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2d9d-1dbd-4236-b405-ffbec4c6f1d7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.115830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.123098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.123704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.124032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.124323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.125833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:25.126135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.127899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa35be-5c5f-4562-b786-b17c1cb81d1f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.148127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 282/563] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:25.148127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.148127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.148127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.156067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.157287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.157791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.158259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.166717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.167599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:25.168643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.168876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:25.183448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:25.184420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 282/564] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:25.190867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.254739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 283/565] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:25.255915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.313040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a35ac-e973-42a1-a202-4226c02ec3ac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.326747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.350534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.351169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.351451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.351726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.361432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.362869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.367234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07184fbf-b051-4aea-8881-39fadec9de1b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:25.376094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:25.377087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:25.378054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:25.379303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 283/566] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:25.380107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.394974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.408803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.409418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.409698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.409974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.410671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:25.410976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.412425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.426424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 284/567] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:25.427691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.488544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86c8e8-03e0-4719-ad5f-bfb81e243126 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.530906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.531539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.532073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.532352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.536224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffc91a-3e0a-4679-836d-c84017784f38 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.542559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:25.548526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:25.549626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:25.550639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 284/568] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:25.551470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.576086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.576662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.576945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.577219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.586107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.587574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:25.592973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:25.594147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:25.595194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 285/569] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:25.596063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.638586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.661861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd6796-f82e-4b9a-81d0-d7ae0a7149c4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.694222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.694827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.695107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.695378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.696102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:25.696201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2373f10-2824-498f-b86a-49815cb8d8b2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.696408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.697873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.715042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 285/570] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:25.715853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.732384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.733025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.733308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.733582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.734473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=06c36d7a8a2246a38fe16e32a4607aae) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:25.734782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.736223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.742384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 286/571] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/role_assignments?user.id=06c36d7a8a2246a38fe16e32a4607aae&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:25.743181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.824415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.836563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d200cf-5b6e-4052-b622-f5700739ca6b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.884873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.885782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.886214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.886640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.894672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960ad62-d796-446f-855d-9bc60b50a0a3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:25.901206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:25.904903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:25.911739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:25.912887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:25.913881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 286/572] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:25.914677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:25.930368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:25.930969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:25.931245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:25.931512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:25.932171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:25.934377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:25.934377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:25.949421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 287/573] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:25.949421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.036698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8deeee8-157d-4798-8c55-24401c60558f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.068923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e517dc9-848c-440f-9a9c-58b710f17471 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.072627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.073247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.073529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.073806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.074468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.074772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.076141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.085041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 287/574] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.085869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.102855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.110950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.111578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.111858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.112130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.124027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.125715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:26.144625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:26.145637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:26.146536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:26.146536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.146536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:26.148756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 288/575] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:26.150159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.177740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.201913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d202f4-0419-4d8e-9a90-bc2870d2742f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.240270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.240963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.241236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.241548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.242211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.242512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.243891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.252447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 288/576] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.253136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.275929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d675eb-dd7e-4391-b178-a79153b9acea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.313226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.314127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.314408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.314684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.326544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.328608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.340915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:26.343830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:26.345101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:26.346187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:26.346187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.346187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:26.346187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 289/577] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:26.346187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.373046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc88e-631a-46d3-ae71-416093ce1ae3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.410988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.411975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.412441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.412751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.419786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.425210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.426702 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:26.441873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:26.442829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:26.443726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:26.443726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.443726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:26.443726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 289/578] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:26.443965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.459461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68e146-8a0f-49c3-a1ae-1d51e05880e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.502652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.503150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.503424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.503691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.504324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.504646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.506081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.518043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 290/579] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.518838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.570127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfe58e-0e20-4832-86c1-b85246f68221 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.599834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.610091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.610774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.611312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.611670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.623766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.625721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.631057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62bcc0-aadd-42da-b775-f627b4eb7d83 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:26.640068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:26.641244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:26.642174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:26.642174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:26.642174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:26.642174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 290/580] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:26.642436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.667808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.674383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.675030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.675321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.675590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.676226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.676525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.677906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.691197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 291/581] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.692029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.760013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6ab86-04c9-4595-a030-f3eb0586b5d6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.802773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.803390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.803671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.803950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.804614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.804930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.806308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.807002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db8785-29b5-47bd-aaa1-a6d9ea3434ea None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.819891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 291/582] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.824967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.853954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.855070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.855705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.856075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.864991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.866557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.873056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:26.881438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:26.882385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:26.883311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 292/583] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:26.883807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:26.943054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca8230-d74a-4eeb-992f-b397a6837876 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:26.944916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:26.984558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:26.985193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:26.985471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:26.985744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:26.986383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:26.986684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:26.988043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:26.999685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 292/584] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:26.999685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.004700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6b91-e8ba-491e-8ac7-e59362109158 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.038674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.039320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.039575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.039885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.047716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.049216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:27.061900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:27.062891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 293/585] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:27.063873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.125168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.131123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15956e72-ccd3-4656-870a-526f268cccf4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.174479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f1385-3b40-4c9e-a823-b59c93745b48 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.174778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.175395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.175675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.175951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.183986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.185472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.192979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:27.198797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:27.199858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:27.200868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:27.204781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 293/586] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:27.205661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.223899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.225004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.225511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.226002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.226972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:27.227605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.229189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.242425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 294/587] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:27.247892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.316286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0545-dfe9-4faf-8e9d-293bf94da9c6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.354063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78f9f3-f22f-4e99-bbbd-dd48b4699367 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.354418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.355018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.355300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.355744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.365899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.367549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:27.383018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:27.384035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:27.385098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:27.391221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 294/588] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:27.391221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.393163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.394175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.394656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.395134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.396463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:27.397026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.399695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.418508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.439628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 295/589] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:27.439628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.486280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.538416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffc09-8106-4836-804e-2fd8ce49463a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.573232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565c239-b9dc-425c-a0f8-7d33b9cafa66 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.581712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.582334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.582617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.582901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.583619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:27.583914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.585388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.609691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 295/590] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:27.609691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.632468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.633169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.633447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.633713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0abda3580f52421d816eac7bee95b492/roles/f239192299174e29a4672c9df068e024` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.649951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0abda3580f52421d816eac7bee95b492, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.651629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.667386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 296/591] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:27 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0abda3580f52421d816eac7bee95b492/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:27.668228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.727489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.754213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa6a20-093d-4114-87c2-55f29045b730 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.793652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:27.795715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.797304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.797588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.797864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.798618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:27.798927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.801748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.831658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b034-36ae-4b38-babd-6e40f42917f2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:27.835671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 296/592] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:27.836888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.882968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:27.883604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.883880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.884164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.900232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:27.901752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b424dbc4-8a47-47dd-bd82-5fe8e6bb228a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:27.918344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:27.919385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:27.920286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:27.920286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:27.920286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:27.920286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 297/593] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:27.920839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24387c-ce25-4ccd-8742-9fee2ed7e688 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:27.921055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24387c-ce25-4ccd-8742-9fee2ed7e688 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:27.921354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24387c-ce25-4ccd-8742-9fee2ed7e688 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:27.926683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 298/594] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:27.926683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24387c-ce25-4ccd-8742-9fee2ed7e688 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:27.982624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1811c-5c5b-48cb-a8e0-0853a4a5cd58 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.027879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.028800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.029379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.029649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.035556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.045347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24387c-ce25-4ccd-8742-9fee2ed7e688 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.050703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.052180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a6f6c996-eb27-463d-9cca-ebc75d714d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:28.069674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.070882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:28.071996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:28.071996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:28.071996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:28.073557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 297/595] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.075001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.075446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.075860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.096457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:28.097694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:28.100478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.107549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.107549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.107549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.107549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.120554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.122133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:28.137913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.138903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:28.139824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:28.139824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:28.139824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:28.139824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 299/596] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.139824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.182151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 298/597] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:28.182843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.274552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21fb29-4523-473b-bbd0-991858c3ee23 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.295809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99a2f8-a582-4ece-a0f4-af93fb550e1a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.315587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.326807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.328034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.328803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.329287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.330785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:28.331322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.332952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.335014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.335628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.335907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.336180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.336995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:28.337180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.338626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.354929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 299/598] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:28.354929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.356215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 300/599] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:28.358311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.424948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.477419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd9d0-3231-4bf1-a608-9e4dcbc25df2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.487658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c3e23-4544-43e6-bc7b-48926c9480b6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.524258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.524969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.525247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.525530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.526232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:28.526575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.528128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.529373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.530250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.530790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.531486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.539281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.540766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.544282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 300/600] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:28.549605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:28.553854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.554828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:28.555737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 301/601] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.556238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.600115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.668539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba1a1-2fd7-4f1a-bfa7-a518dfeb5d0a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.668763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e04770-1031-4049-bc4c-8b46b84cd04e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.670880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.709644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.710596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.710875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.711195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.712320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.712996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.713282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.713555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.720823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.722250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.723996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:28.729409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.730389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 302/602] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.734248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:28.735229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.736303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 301/603] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.737238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.862864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:28.865739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dfa73-7bdb-4b98-8473-bc19179ed970 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.869633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c533-22ea-4843-b389-0d8ba0ed53d9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:28.909832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.910176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:28.910504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.910779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.910995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:28.911094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.911279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:28.911554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:28.912238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:28.912543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.914256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.917896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:28.919322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:28.932031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 302/604] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:28.932031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:28.932258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:28.933140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 303/605] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:28.934067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:28.959009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.044674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6930-f0f9-4e2a-9e01-ea5a6405c403 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.052037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be03726-3774-4be0-b6bf-7ab028750f51 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.084712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.085675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.086176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.086646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.087066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.087682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.087964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.088147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:29.088254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.088764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.088944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:29.089242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.090675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.090782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.099999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 304/606] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:29.101045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.107453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 303/607] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:29.108284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.132038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.195485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.213725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d461b-5715-47e3-aab9-bdaa74c03e1b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.231592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6771f2-3d05-4289-9fc8-4f2f5bc18746 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.255258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.255925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.256199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.256477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/00f82e19c26b45d2a4ac373665f8c9d5/roles/f239192299174e29a4672c9df068e024` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.276940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=00f82e19c26b45d2a4ac373665f8c9d5, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.277190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.277704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.277976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.278250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.278397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.278944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:29.279247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.281365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.303147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 304/608] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:29.303147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.303427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 305/609] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:29 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/00f82e19c26b45d2a4ac373665f8c9d5/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:29.303427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.409814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.454762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d6d64-9668-4949-b545-4f913efbbbe5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.472322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ad367-8bd8-45b6-abc4-4d1fe15fa9da None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.503678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.504679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.505170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.505603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.519835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.521342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.522233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.522961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.523248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.523526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/a7af5dafacef4215b39c06af7c66c77c/roles/f239192299174e29a4672c9df068e024` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.528103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:29.540818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:29.541853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:29.542796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:29.542796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:29.542796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:29.542796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 305/610] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:29.543049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=a7af5dafacef4215b39c06af7c66c77c, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.543134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.545061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.570298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 306/611] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:29 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/a7af5dafacef4215b39c06af7c66c77c/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:29.571187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.713817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf889d8-e3bf-4776-9d24-489f98ebd97d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.727106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590f833-f450-46d8-9627-f70e14067613 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:29.762991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.763467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.764535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.765564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.766382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.767846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:29.768157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.769937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724da998-b8be-47e8-80db-40b2fa329cc3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.775169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:29.775843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.776118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.776456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/f239192299174e29a4672c9df068e024` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.786450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 307/612] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:29.787718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5591e-f20d-4be5-9552-6e5cd43c4193 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.788002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5591e-f20d-4be5-9552-6e5cd43c4193 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.788275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5591e-f20d-4be5-9552-6e5cd43c4193 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.789815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 308/613] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:29.792817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5591e-f20d-4be5-9552-6e5cd43c4193 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:29.799316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0446b5664ae0453a8ccd57dfc562eb9b, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:29.801063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa61b2-c4e4-4ff7-bb3d-06bc20df7dcb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:29.817533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 306/614] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:29 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:29.819104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:29.819389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:29.819660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:29.849761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:29.851365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:29.912558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:29.991610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 307/615] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:30.033193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5591e-f20d-4be5-9552-6e5cd43c4193 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.080667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.081365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.081643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.081910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.092398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:30.095189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:30.108286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:30.109423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:30.110445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 309/616] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:30.118104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.259672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb6263-5915-40c1-81b5-50823b58c77f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.279084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:30.316108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.317242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.317720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.318190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.319682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:30.320211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:30.322862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:30.334800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.380927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 308/617] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:30.390073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.456412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ca04e-d48a-4bb7-8a9e-23c969e086aa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.464247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:30.516641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.517198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.517477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.517748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.550430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:30.555913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:30.556933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:30.557833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:30.564695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 310/618] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:30.567479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.603681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c5963-aa4b-4003-a699-ed48c2d425f6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.685356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.686036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.686323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.686600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.718534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc730a-3913-4e36-8c6c-130455c7f87f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.729521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:30.730874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:30.759502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.760686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.761288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.761772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.762650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:30.763161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:30.764825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:30.770593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:30.771598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:30.772938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:30.772938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:30.772938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:30.772938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 309/619] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:30.781921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.782292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 311/620] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:30.785230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:30.825650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:30.893169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:30.895591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32497e-6b5f-4085-b57e-493ae2423703 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.932159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:30.933196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:30.933698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:30.934177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:30.945696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:30.947450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:30.959664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2acc3-0487-46dd-8523-8c2c577ac67d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:30.963301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:30.964253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:30.965239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:30.965239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:30.965239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:30.972702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 312/621] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:30.974272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.027574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.028203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.028486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.028786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.029435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:31.029742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.032076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.059053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 310/622] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:31.066943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.068967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:31.089164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cf83e-0be6-4ca8-964c-286dccf66b8d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.177302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.177929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.178211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.178491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.179145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:31.179449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.180952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.201009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:31.211565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 313/623] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:31.217301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af749f39-7f5c-4af3-84ee-5fda5f9b9bc4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.221186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.301321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.302350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.302846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.303331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.319239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.321801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:31.346987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:31.347994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:31.349514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 311/624] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:31.354044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.386918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcbeac-e7cb-43a3-af6d-ac6f1c544e99 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.428010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.429076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.429591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.430066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.438804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.440600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.443613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:31.454861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:31.455901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:21:31.456902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:31.459511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 314/625] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:31.464610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.584993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7875-49e5-439e-9a6b-42cb509338e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.623031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c2e1-0489-486f-8bb8-db4521955852 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.626228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.627095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.627619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.628101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.629154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:31.629690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.631351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.654219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 315/626] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:31.661957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.716556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.717199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.717487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.718235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.719222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:31.719927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.721917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52cc1e-2b86-4e0c-87b0-79415c2b3af8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.724058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:31.734462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:31.739264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 312/627] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:31.741018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1134c-7fe5-4e85-833c-ed2a09be7c93 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.741475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1134c-7fe5-4e85-833c-ed2a09be7c93 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.741816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1134c-7fe5-4e85-833c-ed2a09be7c93 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.744142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 313/628] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:31.754047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.754501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.754878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.774138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:31.775584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:31.778319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33ebca-e7b5-441e-b77b-0e54b48b7e26 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:31.825891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:31.826755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:31.827035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:31.827359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:31.866724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 314/629] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:21:31.867963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:31.924501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:31.926450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:31.994567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:32.001025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8a3c2-93d6-48a0-af01-ba59d3092f95 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.032299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:32.039662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.040658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.041157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.041645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/06c36d7a8a2246a38fe16e32a4607aae/roles/f239192299174e29a4672c9df068e024` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.057867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=06c36d7a8a2246a38fe16e32a4607aae, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:32.058192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 316/630] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:32.059864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:32.065007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.075675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 315/631] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:31 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/06c36d7a8a2246a38fe16e32a4607aae/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:32.175868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078f06-4ad6-4b5d-9c8f-1948a76210a4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.220071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.220751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.221035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.221308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.232453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:32.237865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:32.256751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:32.257766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:32.258706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:32.258706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:32.258706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:32.261008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 317/632] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:32.267448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.378802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.392696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:32.451602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcbd59-e69e-4560-a6b7-1fcdbfa08f8d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.498669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.500207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.500791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.501286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.502424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:32.503850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:32.505472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:32.519530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 316/633] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:32.528917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.575003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9158e-b91d-4d91-be3f-c66ff9c961fb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.628430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.629154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.629443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.629718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.638369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:32.648457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-716bb8a5-cd51-460c-a8e5-5b5277eb069b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:32.657539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:32.658523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:32.659383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:32.662936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 318/634] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:32.668458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2b7d-fd8b-44f7-b3d3-b8292b206af6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.668458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2b7d-fd8b-44f7-b3d3-b8292b206af6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.668458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e2b7d-fd8b-44f7-b3d3-b8292b206af6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.670331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 319/635] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:32.672265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e2b7d-fd8b-44f7-b3d3-b8292b206af6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.755857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07b1aa-4c0e-485f-9960-49f1e32e2eac None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.760541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:32.807023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e2b7d-fd8b-44f7-b3d3-b8292b206af6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:32.816394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.817399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.817873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.818378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.828319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:32.830264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-eb4ba3f8-7e26-4844-9d5a-82b318d62851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:32.845376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:32.846352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:32.847261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 317/636] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:32.848416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.848416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.848416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.865295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:32.865904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.866178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.866445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.867108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:32.867407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:32.867971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:32.868798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106ff8f-e614-4585-8059-9bc03e360ff6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:32.869189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 320/637] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f2d8-50e3-40f2-9069-6184bc8a27b5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f2d8-50e3-40f2-9069-6184bc8a27b5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f2d8-50e3-40f2-9069-6184bc8a27b5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 321/638] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:32.882105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3f2d8-50e3-40f2-9069-6184bc8a27b5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:32.960727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 318/639] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:32.961640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.002817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3f2d8-50e3-40f2-9069-6184bc8a27b5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.004305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.038230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.044815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.045427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.045708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.046763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.047295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:33.047687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.049222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3d61a-7396-45c8-a6d3-941e75771320 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.064365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 322/640] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:33.065720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.066060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.066387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.086938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:33.088359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:33.097247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a7f49-7c23-4211-9d54-4e2d11dae395 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.135604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.136244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.136526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.136905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.148904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.150581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:33.166297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:33.167255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:33.168150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:33.168150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:33.168150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:33.168150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 319/641] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:33.173485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.181209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 323/642] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:33.182325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.304305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.311534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1fab-f04c-423e-92dd-608edcbc25a5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.333761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f169f-d5b2-443d-b94e-542e3cc3f79c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.338919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.352908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.353778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.354207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.354633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.355434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:33.355874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.357439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.370571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 324/643] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:33.371677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.389450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.390656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.391007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.391489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.392860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0abda3580f52421d816eac7bee95b492) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:33.393414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.396034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.412616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/role_assignments?user.id=0abda3580f52421d816eac7bee95b492&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:33.412616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.542015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04928a3e-1fde-4b2a-a770-7bce5be4a58b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.545142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3568e07-7a0b-44b9-9eff-f639504599e0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.582075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.582617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.582970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.583169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.587029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.587786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.588162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.588566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.596323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:33.598558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.600016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:33.601681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:33.602711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:33.603623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 325/645] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:33.604351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:33.619674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:33.620725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:33.621825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 321/646] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:33.622474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.632495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.669131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.741606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7af86-5be8-4af6-950e-7e9960c28014 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.765263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546b1e-7394-45a2-94fa-8711f7d32947 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.784231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.784899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.785161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.785456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.801394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:33.808164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:33.809330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:33.810202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 326/647] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:33.810908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.813140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:33.814314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:33.815833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:33.816329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:33.817326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:33.817879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:33.819476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:33.831613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 322/648] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:33.833002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:33.957061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.973863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8c5d4-0d8a-4431-9325-2a6aa6fd9f1c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:33.976508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.004460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80840a78-a9df-4296-920b-5a3e0e8c3e04 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.020385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.021290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.021899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.022313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.023593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:34.024294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.026078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.047442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.048222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.048650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.049203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.049976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:34.050427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.051948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.059462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 327/649] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:34.061009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.062943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 323/650] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:34.069896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.176765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2074-08e4-4baa-9025-09a8cb45542c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.181019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072009c4-3a6d-485b-9982-81acba3aab1a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.208472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.215867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.216527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.216834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.217267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.224546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.225547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.226025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.226466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.229980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.231454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.238840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.240783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:34.248147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:34.249757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:34.250850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:34.250850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.250850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:34.255426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 324/651] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:34.256319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:34.262481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:34.263460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:34.264406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:34.264406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.264406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:34.272045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 328/652] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:34.273356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.290975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.375167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86b98e-f644-4e54-900d-2a7d0805aba7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.403650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b277-a96c-4c18-9656-7da8d64f6035 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.410279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.410897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.411180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.411459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.447155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.447678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.447953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.448222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.448912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:34.449212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.450583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.463978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 329/653] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:34.465209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.471665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.492935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.494398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dcf4e-697d-4650-a240-73552aefbad4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.542615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.576571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f738ef-4764-4d45-8383-2440526826f4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.611081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 325/654] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:34.612405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723810-57c6-40b6-87b6-5ae3e82805e6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.612718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723810-57c6-40b6-87b6-5ae3e82805e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.613014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723810-57c6-40b6-87b6-5ae3e82805e6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.614453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 326/655] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:34.617210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.617829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.618335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.618455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.619015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b723810-57c6-40b6-87b6-5ae3e82805e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.619128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:34.619398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.620806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.633392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 330/656] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:34.634227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.724322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.746869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b723810-57c6-40b6-87b6-5ae3e82805e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.753437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc7a4f-c2f2-4b1e-bbb6-ca47a95e2c5a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:34.785013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.785655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.785928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.786197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.786326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:34.792103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:34.792754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.793031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.793296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.794438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.796866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.805588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:34.807047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-63f732d2-3fd4-4ff2-b7e4-07c84ade1a88 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:34.809780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:34.810925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:21:34.813075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:34.816182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 327/657] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:34.817462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:34.817743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:34.818016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:34.826861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:34.828161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:34.829380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:34.829380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:34.829380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:34.830795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 331/658] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:34.831721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.842479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:34.844129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:34.928634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 328/659] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:34.928634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:34.985835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8ac7-46c8-409a-9d83-4f200b77f490 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.025276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.036645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.037883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.038367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.038845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.049787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.051529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:35.067004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:35.068225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:35.069417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:35.071855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 332/660] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:35.072991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.078294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d0e41-ab7b-4cb1-9564-e3277232f609 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.117626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.118453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.118827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.119198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.120064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:35.120541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.122941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.138664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.148462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 329/661] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 29 21:21:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:35.148462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.217120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e69641-6226-44b0-87fe-197e76fc3c16 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.261828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.262360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.262639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.262913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.263562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:35.263866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.264084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9cc17-d638-47a2-8807-a30ed8045c25 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.265796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.278452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 333/662] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:35.279314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.307045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.313983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.314641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.314926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.315200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.315942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:35.316252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.317705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.337758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 330/663] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:35.338632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.404239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f296863-bdfd-4a26-b693-de6dd6b47c48 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.408406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.443937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.444552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.444884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.445314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.454833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760048-6708-425b-a53c-305cb77a9c62 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.496561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.497242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.497522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.497793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.510266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:35.516980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:35.517290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:35.518206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:35.521585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:35.522244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.522319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 331/664] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:35.522319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.581584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.623066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 334/665] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:35.624687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.629185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.633982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd92fb7-0a2a-4364-addc-debadd66f8ea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.672342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.673005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.673418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.673570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.680986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.682426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:35.695258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:35.696213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 332/666] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:35.697181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.740273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ac54f-e607-4953-8b92-8584d24a7c41 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.776893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.777493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.777774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.778047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.805273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfbaa0-50b3-49c6-911e-b808a65e6211 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.814865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.846683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:35.847395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.847680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.847949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.848625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:35.848929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.850284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c3a9e-bdf4-4bfe-b61d-6c978e387709 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.854089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:35.855522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:35.858564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 333/667] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:35.859369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:35.860542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77940-186c-4244-a49f-efddf4a1ca41 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:35.861938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77940-186c-4244-a49f-efddf4a1ca41 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:35.861938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa77940-186c-4244-a49f-efddf4a1ca41 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:35.862838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 334/668] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:35.867683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa77940-186c-4244-a49f-efddf4a1ca41 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:35.976286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 335/669] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:35.976578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa77940-186c-4244-a49f-efddf4a1ca41 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:35.977155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.016445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.017106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.017390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.017665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.029534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.030992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:36.046462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:36.047609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:36.048616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:36.048616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.048616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:36.048797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 335/670] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:36.049527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.077235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.086927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f35c9f-f233-4fc6-a2ad-eec23b1245fa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.089249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.123018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.123616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.123892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.124770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.125474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:36.125773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.127204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3868a93-9b7e-446c-a397-6e0653b823f9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.138752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 336/671] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:36.139981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53360626-3014-4a64-ac67-5544b652f81a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.140263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53360626-3014-4a64-ac67-5544b652f81a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.140534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53360626-3014-4a64-ac67-5544b652f81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.161779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-53360626-3014-4a64-ac67-5544b652f81a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:36.162967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-53360626-3014-4a64-ac67-5544b652f81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:36.168015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58080c1b-b377-4151-9a5f-5933ac00186c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.206863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.207490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.207764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.208031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.220060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.221667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.237321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 337/672] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:36.238174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53360626-3014-4a64-ac67-5544b652f81a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:36.239070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:36.240778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:36.241688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:36.241688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.241688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:36.241688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 336/673] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:36.241688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.335429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.364404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.364658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e19277-e1a3-4b69-b609-8eba6fbf0944 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.375119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53360626-3014-4a64-ac67-5544b652f81a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.401053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.401983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.402383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.402660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.403427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:36.403729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.405138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.415158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.415801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.416061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.416336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.422868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 337/674] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:36.423503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.427957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.429538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:36.445743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:36.451945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:36.453738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:36.453738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:36.453738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:36.453738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 338/675] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:36.453738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.542311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147d18a-7184-4923-bf98-e2ec4956a180 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.570415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a11ca-9081-49ab-85ba-dbe9c710778f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.583135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.583764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.584041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.584310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.585069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=00f82e19c26b45d2a4ac373665f8c9d5) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:36.585373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.586857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.588658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.600821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 338/676] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/role_assignments?user.id=00f82e19c26b45d2a4ac373665f8c9d5&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:36.601785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.606866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.607494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.607776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.608051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.608794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:36.609147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.610558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.627714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 339/677] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:36.628610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.638833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.744500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0b996-dad5-4266-bc09-cc0fc6ef41d9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.758349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8740946-c2c1-4096-8a1f-4943ee0a57a2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.787514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.788148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.788424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.788726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.797823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.799304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.803538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:36.804194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:36.804483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:36.804809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:36.805465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:36.805775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:36.807146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:36.813389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:36.814633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:36.815574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 339/678] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:36.816083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.831959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 340/679] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:36.833010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:36.903694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.913746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:36.959384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c0351-3471-41e8-9717-6f361f6fbc69 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:36.963072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7722-4088-4fe6-aff2-67a5cca4b902 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.003529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.004155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.004438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.004811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.006941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.007982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.008388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.008839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.016468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.017988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.018183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.019577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:37.032624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:37.033642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:37.034620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:37.035329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:37.036202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:37.037176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:37.037176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:37.037176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:37.037350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 341/680] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:37.037350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.037477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 340/681] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:37.038431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.155051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836081b-7f12-4d1e-9f63-6cd7885f0965 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.158211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df032ea-6e35-4c56-87d3-1e44ce80f057 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.170379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.185680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.195158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.195497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.195796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.196070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.196169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.196334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.196439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.196698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.197389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:37.197690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.199638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.203354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.204809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.217235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 342/682] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:37.217235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:37.217468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:37.218837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 341/683] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:37.219790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.338529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c325fc2-d109-4b32-8a48-484d4d667561 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.338975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf517b1-9485-4061-9e23-4f9019be6110 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.378616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.379219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.379496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.379773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.380397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:37.380733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.382089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.387762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.388370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.388668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.388942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.390220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:37.390529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.391936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.395054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 343/684] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:37.396545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.415258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 342/685] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:37.416011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.446359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.461118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.526667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075be22d-66a3-4484-bd70-888a7c452244 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.543735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b8cab-455e-4a71-b695-022081f5c19a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.570184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.570793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.571104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.571333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.572017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:37.572317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.574017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.586243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 343/686] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:37.587087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.597408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.598058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.598385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.598656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.599390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=a7af5dafacef4215b39c06af7c66c77c) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:37.599694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.601207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.612640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 344/687] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/role_assignments?user.id=a7af5dafacef4215b39c06af7c66c77c&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:37.612640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.708034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.720630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc0a2-f6f1-44a6-806c-bd0f7b493657 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.761495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361bee-c61c-433b-bebb-2514c0c7a7f2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:37.769209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.769953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.770242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.770513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.771170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.772682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.774614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:37.803743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 344/688] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:21:37 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:37.804392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:37.823548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:37.824683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:37.825428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:37.826589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:37.937584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:37.938875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:37.966692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e212086-176b-4871-adfb-245bb4f063e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:38.016910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:38.017710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.018087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.018461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.019299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:38.019720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:38.021302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:38.028825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 345/689] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:38.076378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 345/690] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:38.083296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:38.102013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:38.210718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d87b-b46a-41e8-922a-70ed36e8205c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:38.250884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:38.251462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.252150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.252436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.264131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:38.266200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:38.283148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:38.284174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:38.285128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:38.285128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:38.285128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:38.286023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 346/691] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:38.291614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:38.308710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:38.409160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2bf42-0e8e-4169-b5f8-fa0f777612da None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:38.446937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:38.447552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.447832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.448107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.448786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:38.449092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:38.450468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:38.463442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 346/692] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:38.469481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:38.536609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:38.584891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81677906-9a8e-4721-8771-8117e9fa72d0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:38.621250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:38.621893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.622178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.622453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.630426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:38.631890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a0b3b572-83f1-43fc-9438-1fbaec85ab69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:38.644276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:38.645218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:38.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 347/693] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:38.652943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:38.659385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b3dbe-2544-4441-954a-ee8d645a190b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.659670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b3dbe-2544-4441-954a-ee8d645a190b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.659989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b3dbe-2544-4441-954a-ee8d645a190b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.662599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 348/694] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:21:38.666704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.666985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.667260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.685861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:38.687585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:38.773178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 349/695] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:21:38.824991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:38.871405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ecefd-bca3-4576-a676-ab13769ec0e0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:38.928823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:38.929439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:38.929727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:38.930003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:38.930705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:38.931012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:38.932454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:38.954388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 347/696] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:38.965688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:39.055331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:39.107414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:39.123789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe00d-2515-4c74-ad92-d991cf1d8366 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:39.165918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:39.166532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:39.166809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:39.167079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:39.167869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=06c36d7a8a2246a38fe16e32a4607aae) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:39.168177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:39.169951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:39.177134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 350/697] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:38 2022] GET /identity/v3/role_assignments?user.id=06c36d7a8a2246a38fe16e32a4607aae&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:39.332846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635103a-1909-4ea2-bfde-ffb70687c4e9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:39.384648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:39.401438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:39.402480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:39.402980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:39.403463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:39.404254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:39.406727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:39.418547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:39.419662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:39.420607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:39.424119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 348/698] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 29 21:21:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:39.431021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:39.557117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c551c56-998f-475a-aa2b-aae541c574de None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:39.576170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:39.599054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:39.599623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:39.599910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:39.600186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:39.600905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:39.601191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:39.602687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3dda-017d-4160-ae89-315c81aa3083 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:39.612936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 351/699] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:39.614413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:39.759821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4470a6-754b-4dce-b564-e2c6ee8260c9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:39.800327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:39.801358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:39.801863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:39.802333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:39.803173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:39.803669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:39.805443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5471e4-f2f5-4387-b9aa-6793ddaffc1f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:39.813125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 349/700] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:40.083001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758ac0a-6220-44ac-ac7d-9e4652c8fe18 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:40.084042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758ac0a-6220-44ac-ac7d-9e4652c8fe18 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:40.084535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758ac0a-6220-44ac-ac7d-9e4652c8fe18 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:40.086440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 352/701] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:40.091701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feffaf-28a4-4340-8bc4-4556608c1fd7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:40.092427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feffaf-28a4-4340-8bc4-4556608c1fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:40.092427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feffaf-28a4-4340-8bc4-4556608c1fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:40.112826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c3feffaf-28a4-4340-8bc4-4556608c1fd7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:40.114040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c3feffaf-28a4-4340-8bc4-4556608c1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:40.189073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:40.211071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 350/702] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:21:40.391725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad800a6-25be-4e8d-91b7-e38d3db84bf3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:40.391905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad800a6-25be-4e8d-91b7-e38d3db84bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:40.392179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad800a6-25be-4e8d-91b7-e38d3db84bf3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:40.393763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 353/703] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:40.399061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:40.399344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:40.399615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:40.418773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:40.419948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:40.519146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 351/704] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:40.674431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad800a6-25be-4e8d-91b7-e38d3db84bf3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:40.786668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad800a6-25be-4e8d-91b7-e38d3db84bf3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:40.822048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:40.822697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:40.822979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:40.823257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:40.823850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:40.825244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:40.837436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:40.838530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:40.839404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 354/705] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:40 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:40.845448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:40.900170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:40.957959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:40.986265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c186a4-00d4-48b2-9dbd-f8e5f21c5229 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.008920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:41.028503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.029147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.029430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.029704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.030382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] RBAC: Adding query filter params (name=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:41.030505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c05e86-b7f7-4305-b2e9-4d2ef01d1126 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.030688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:41.033092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440da6c-813d-4c0f-97e8-295ec81e331a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:41.042641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 352/706] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:41.043983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970acbb2-26e8-46d0-ad18-7b9809e0fbcb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.044267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970acbb2-26e8-46d0-ad18-7b9809e0fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.044542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970acbb2-26e8-46d0-ad18-7b9809e0fbcb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.046444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 353/707] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:41.047548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970acbb2-26e8-46d0-ad18-7b9809e0fbcb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.069598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.070603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.070883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.071420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.085322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-619b4b0c-e4d1-4340-8b9c-434b129ad559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:41.091857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:41.093010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:41.094882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 355/708] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:41.096444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.096444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.096444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.115557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:41.116903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:41.183683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970acbb2-26e8-46d0-ad18-7b9809e0fbcb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.205026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 356/709] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:41.206227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.226441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.227149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.227482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.227747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.228442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] RBAC: Adding query filter params (type=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:41.228812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:41.230752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:41.236757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 354/710] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:41.278653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:41.388782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a025a6-65df-4466-b441-3844fe00e162 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.440195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.440849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.441132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.441407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.442080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:41.442383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:41.443749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:41.457367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 357/711] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:41.462605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.535452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:41.594090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd540d-7746-4375-8f48-28b6ace3e325 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.605074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.636346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.637012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.637439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.637839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.652687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:41.654207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:41.674718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:41.676950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:41.678151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:41.678151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:41.678151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:41.678151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 355/712] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:41.681554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.731505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:41.743173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72660bf5-11a4-42b7-b7ce-964554544d9e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.781278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.782065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.782346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.782628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.800921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-0ade1559-97db-4d1c-8a89-c6b083f7cefc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:41.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:41.808768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:41.810887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 358/713] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:41.812923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:41.819945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18beb-3c52-41f5-82b5-fbf38ce290dc None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.819945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18beb-3c52-41f5-82b5-fbf38ce290dc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.819945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef18beb-3c52-41f5-82b5-fbf38ce290dc None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.819945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 359/714] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:41.819945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef18beb-3c52-41f5-82b5-fbf38ce290dc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:41.820832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc65481-fe22-4b44-8de4-c219b2212f7d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:41.867427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:41.867427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.867427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.867427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.868176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:41.868548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:41.870133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430989b5-7383-4428-bf5d-f8f7035652c2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:41.884862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 356/715] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:41.886897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:41.887252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:41.888019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:41.911174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:41.913223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:41.966035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef18beb-3c52-41f5-82b5-fbf38ce290dc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.014186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 357/716] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:42.016042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.016471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.017227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.017512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.017784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.018457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:42.018760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.020625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.038886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 360/717] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:42.041734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.067483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.156985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.174419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fa9d2-506f-401e-b732-cc4c114c5acf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.206222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502aae1-3078-4a7f-9c9d-e29fe00bd2dc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.230961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.232796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.233305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.234234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.243902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.244550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.244855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.245124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.246211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.248658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.256487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.259348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ddf8a1f5-efb1-49c9-ae8a-c48734651337 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:42.263740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:42.265303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:42.266377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:42.272998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 358/718] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:42.274979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b335-1335-4967-aa99-6219595a1069 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.275452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b335-1335-4967-aa99-6219595a1069 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.275566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b335-1335-4967-aa99-6219595a1069 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.277290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 359/719] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:42.277830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b335-1335-4967-aa99-6219595a1069 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7d18484b-3948-4b35-b0b3-cca36332ab0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:42.293737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:42.294778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:42.295819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:42.295819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:42.295819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:42.298556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 361/720] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:42.300069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.300618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.301101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.319820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:42.321343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:42.392493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.408618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 362/721] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:42.408618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.416376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b335-1335-4967-aa99-6219595a1069 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.466218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.466822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.467103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.467377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.468137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:42.468525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.470615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.495594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 360/722] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:42.495594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.506941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.533565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd35ec2-52f1-4848-b8ef-5e59e78537b9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.574582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.575116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.575399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.575675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.576378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:42.576727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.578950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.592815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 363/723] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:42.593913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.634979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272df293-bbdc-45a4-bd7b-de4497cccede None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.666282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.686108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.687278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.687557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.687826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.699179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:42.704629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:42.705601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:42.706486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 361/724] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:42.707241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.721271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74b47f-b23e-449b-96ae-25182916ee61 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.758788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.759412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.759689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.760108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.788886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.829221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243f3a-ead1-48c2-8464-2644a41135f1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:42.842213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.843779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.882547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:42.883164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:42.883444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:42.883711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:42.895127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:42.897545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:42.908131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:42.913555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:42.914544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:42.915615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 362/725] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:42.916232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:42.958684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 364/726] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:42.959441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.046257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60b116-9d32-435c-99ba-a7a7981d8af7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.072831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.078635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899fcc5-4134-4b6f-b9b4-8e6d48a576f8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.083107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.083639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.083919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.084196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.084891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.086266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.102921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 363/727] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:21:42 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:43.103601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.157464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.158477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.159604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.160128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.161540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:43.162350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.164539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.184386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 365/728] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:43.184386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.211826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.249530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fed0a2-0d38-4606-bc19-75c10f8a91d6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.299074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.299696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.299971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.300240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.301507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:43.301507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.302848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.327859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 364/729] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:43.329254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.340668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b80408-7587-4b69-8835-3b3cb7828f82 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.378982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.379627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.379911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.380189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.380890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.398672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.400475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:43.419821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:43.421102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:43.422053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:43.422053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:43.422053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:43.426887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 366/730] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:43.427973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.446574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea8eae-d0ca-4e45-a829-33f8040f3c1a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.484320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.486012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.486841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.487321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.499304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.501186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.504360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:43.520903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:43.521964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:43.522865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:43.522865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:43.522865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:43.525129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 365/731] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:43.526151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.549577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941da3-5f74-438b-aa3e-79cab9733f31 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.590292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.591015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.591237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.591518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.638101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.646139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb4749-e9b0-4271-a6a8-40d2a2fb1f9f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.668780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.670339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a5beb-13c6-446a-a40e-08a893c1518e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.683247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343455-d3ac-4830-8299-921e79326843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.683960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343455-d3ac-4830-8299-921e79326843 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.684300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343455-d3ac-4830-8299-921e79326843 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.684659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343455-d3ac-4830-8299-921e79326843 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.685379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67343455-d3ac-4830-8299-921e79326843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:43.685728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67343455-d3ac-4830-8299-921e79326843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.687683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67343455-d3ac-4830-8299-921e79326843 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.704686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 366/732] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:43.704686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343455-d3ac-4830-8299-921e79326843 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.744453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.783372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 367/733] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:21:43.785035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16c13-9902-40cf-a202-c1117ff2aa92 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.785390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16c13-9902-40cf-a202-c1117ff2aa92 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.785739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e16c13-9902-40cf-a202-c1117ff2aa92 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.788683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 368/734] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:43.788683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e16c13-9902-40cf-a202-c1117ff2aa92 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.825758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343455-d3ac-4830-8299-921e79326843 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.868875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.869558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.869917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.870270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.870927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:43.871229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.872651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.888733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 367/735] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:43.889582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:43.911361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:43.927189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e16c13-9902-40cf-a202-c1117ff2aa92 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:43.964630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:43.965375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.965974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.965974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:43.973586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:43.975029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d564a2cc-7c4a-4b5b-8c5e-cd49a93b636c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:43.988090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:43.989256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 369/736] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:43.990406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:43.991109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.012048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ef1c7-3d03-4279-ac3a-2212a7946c48 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.014161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:44.016226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:44.044922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.053279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.054027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.054312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.054589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.068626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.070100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:44.085220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:44.086247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:44.087170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:44.087170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:44.087170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:44.090943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 368/737] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:44.091805 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.105127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 370/738] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:21:44.117726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.184339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.231307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af0e9-c93e-4ccb-acd3-79afef019e65 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.282372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1b76-f49b-4b0b-9f3f-17187208d56f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.283757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.287403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.288193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.289088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.305393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.307485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.324631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:44.325252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:44.326373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:44.328248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:44.328983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.328983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.329283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.330077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:44.330477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.331108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 369/739] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:44.332099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.337309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.351318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 371/740] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:44.352288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.374440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.461458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3036-517d-4893-b001-6f19d49e679c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.467320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b789d0d-9685-4722-81e0-685e820c9920 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.490753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.501224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.501834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.502113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.502385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.503028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:44.503354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.504188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.504755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.504920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.505154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.505424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.506162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:44.506461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.507894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.522803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 370/741] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:44.522803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.533372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 372/742] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:44.538430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.624311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.649224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a833d-5191-4cfd-a16d-e517ebfeef89 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.656161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73f547-bb75-454a-9a41-aa7eb8875b49 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.687454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.688077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.688361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.688676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.689498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.691135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.700379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.701019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.701386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.701590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.702911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0af35a01c7c6427681a1f12508c1ff35) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:44.703314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.704762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:44.721030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 371/743] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/role_assignments?user.id=0af35a01c7c6427681a1f12508c1ff35&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:44.721030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.721678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 373/744] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:21:44 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:44.724991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:44.838914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:44.914111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673af8-b397-4936-a180-15a2b9e9247a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.965109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e51738-ba1c-4143-9db7-754c2ec5b5ad None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:44.975329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:44.976419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:44.976981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:44.977515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:44.986796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:44.988348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.001965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:45.006225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:45.007162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:45.008081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:45.010339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 372/745] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:45.012764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:45.013786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.014215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.015274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.017768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:45.109926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:45.111406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6031d43-e7e6-43e5-b5c8-c4211ba73b0b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.143903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faa459-2cc9-4430-b638-b12726abbcbb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:45.161670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:45.195227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:45.195758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.196040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.196358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.197462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:45.197778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:45.199300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790665cc-b655-4da1-a694-1df590723eaa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.220652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 373/746] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:45.220652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dccb-9c6a-4617-a00c-7a1320ed15bd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.220652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dccb-9c6a-4617-a00c-7a1320ed15bd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.220652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67dccb-9c6a-4617-a00c-7a1320ed15bd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.220652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 374/747] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:45.224986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67dccb-9c6a-4617-a00c-7a1320ed15bd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:45.277610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 374/748] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:21:45.278852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.279144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.279423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.304362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:45.305654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:45.336514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:45.358742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67dccb-9c6a-4617-a00c-7a1320ed15bd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:45.386652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 375/749] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:45.388078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:45.399100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:45.400187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.400707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.401212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.402214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:45.402829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:45.404533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.416602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 375/750] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:45.507523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da9bc4-3416-44e9-95bd-ba503eb50cfa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:45.667246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:45.667959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.668239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.668513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.680923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:45.682375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:45.698673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:45.699704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:45.701272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:45.701272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:45.701272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:45.701272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 376/751] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:45.705781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:45.766104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:45.830434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:45.848268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:45.878613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e800e76-4841-4c64-9e73-e04a244bd298 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:45.950050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:45.953128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:45.953563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:45.953978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:45.954775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:45.955414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:45.957988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:45.979761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 376/752] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:45.980756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.043688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448378-0f4c-4981-bb35-18e9a616cc61 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.118623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:46.128378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.129127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.129413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.129690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.130308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.131764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.132251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d60b83-f0db-499a-b6e8-18edcdf667d9 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:46.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:46.155979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:46.157125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:46.159657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 377/753] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:46.169716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.173997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.174702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.174990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.175592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.184683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.186278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:46.200933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:46.203119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:46.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 377/754] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:46.208744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.322065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba95b0-1f7d-4bb6-8b65-3d9dbdc0b3f0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.359595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.360225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.360483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.360814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.361525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:46.361839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.363288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.364935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:46.380570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 378/755] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:46.387838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.432957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:46.434066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b3409-0c21-4536-8af2-60230998ae8a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.520450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.521442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.521899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.522370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.523177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:46.523657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.525225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.540037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 378/756] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:46.540635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb742d5-3743-4500-9ab5-4fa22682fbd1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.546098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.582239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.582812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.583095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.583368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.584111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=06c36d7a8a2246a38fe16e32a4607aae) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:46.584422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.585930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.592354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 379/757] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/role_assignments?user.id=06c36d7a8a2246a38fe16e32a4607aae&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:46.733251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081424e6-704f-490c-9edd-f2bd8bbd9746 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.783982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.784655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.785230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.785527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.786380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:46.786693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.788180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.796604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 379/758] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:46.802733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.886930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:46.913370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df847-4da9-4ec2-ab18-fc53c8287d6f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:46.953432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:46.954067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:46.954530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:46.954810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:46.955523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] RBAC: Adding query filter params (interface=public, service_id=5cf92664779f49cc8ad17d14c5c3b97b, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:46.955831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:46.957265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb22c6-0972-4074-9654-5c748e650ab4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:46.965671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 380/759] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/endpoints?service_id=5cf92664779f49cc8ad17d14c5c3b97b&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:46.975242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:46.986095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:47.118252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867647c-0d90-498a-9505-e6b9b8a3bec4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.118427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867647c-0d90-498a-9505-e6b9b8a3bec4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.118744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867647c-0d90-498a-9505-e6b9b8a3bec4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.120370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 381/760] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:47.122519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b14d31-e224-4fe0-a41f-ba7b2ac0f845 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:47.126006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d784e-0e1e-42af-b691-1070397cb2a1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.126286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d784e-0e1e-42af-b691-1070397cb2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.126558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d784e-0e1e-42af-b691-1070397cb2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.160929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:47.162149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.162149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.162326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.162823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:47.164193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4fcf3d-254a-4e5c-86d6-3a9beb7a134f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:47.170175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 380/761] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:21:46 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:47.177898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-542d784e-0e1e-42af-b691-1070397cb2a1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:47.179154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-542d784e-0e1e-42af-b691-1070397cb2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:47.293835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 382/762] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:47.400437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410e88-4754-4c61-b16b-d9ec49cc4009 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.400956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410e88-4754-4c61-b16b-d9ec49cc4009 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.400956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11410e88-4754-4c61-b16b-d9ec49cc4009 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.402280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 381/763] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:47.408485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.408991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.409072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.429442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:47.430700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:47.506738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:47.513139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 383/764] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:47.596744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11410e88-4754-4c61-b16b-d9ec49cc4009 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:47.672666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:47.708018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11410e88-4754-4c61-b16b-d9ec49cc4009 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:47.746816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:47.747459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.747813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.748090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.761339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:47.767157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:47.768132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:47.769034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 382/765] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:47.775158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:47.824493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:47.935725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f9881-1560-4943-addc-24460fe872ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:47.953253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c0f1-fa12-44d8-a477-de0256d06c7b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:47.988401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:47.996536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:47.996949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:47.997739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.997878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:47.998218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.998524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:47.999307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:47.999790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] PATH_INFO: `/v3/services/network` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.000555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:48.000750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.001199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.002495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.003945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e90ac1b0-de61-425a-8114-b37ebe19d7cf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:48.020779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:48.021840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:48.022704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 383/767] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:47 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:48.023518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff92b2-4982-4224-974e-e5c946070f1b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.023518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff92b2-4982-4224-974e-e5c946070f1b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.023518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff92b2-4982-4224-974e-e5c946070f1b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.024306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 384/768] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:48.027179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 384/768] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:48.028205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.031775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.032063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.032341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.057054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:48.058077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:48.108870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:48.151816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 385/769] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:48.153354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.174294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c595a-8011-45a2-931c-600338282ad3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.247535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.250293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.250818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.251483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.277255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.279291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:48.306163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:48.307192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:48.308227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:48.308227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:48.308227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:48.311769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 385/770] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:48.321618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.358973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c1c6-7829-42bf-8d86-3515c749f774 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.411265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:48.437760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.438392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.438674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.439003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.439592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] RBAC: Adding query filter params (name=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:48.439975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.441442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.453611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 386/771] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:48.461986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.526082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3c599-ecc2-42d5-b03b-d6dafab77bbf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.549216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:48.580123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.581102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.581795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.582216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.583040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:48.583782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.585766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.599588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514339-ca99-4c22-a076-3b15675faf2a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.606172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 386/772] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:48.607303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.653615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.654438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.654834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.655439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.656186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] RBAC: Adding query filter params (type=network) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:48.656843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.658765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.664749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 387/773] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:48.666031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.736031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:48.751283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eb0ac-4c13-40dd-b880-65227dbee597 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.796518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.797935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.798598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.798988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.803295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cfdeb-bc4b-4a00-b843-cd0bccba87eb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:48.810763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:48.816850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:48.818164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 387/774] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:48.819045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.852926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:48.854072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:48.854928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:48.862216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:48.865079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:48.874623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:48.876296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:48.896440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:48.897723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:48.898901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:48.906016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 388/775] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:48.908279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:48.969378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b36ddf-cac2-45c0-8aeb-b739f215dad4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.028370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.029128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.029406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.029875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.031014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] RBAC: Adding query filter params (interface=public, service_id=95948328202a4f36a69f9a3be5de711c, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:49.031596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.033311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.033727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d614-7af0-4163-b848-fe0be0bc814a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.044464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 388/776] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/endpoints?service_id=95948328202a4f36a69f9a3be5de711c&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:49.046684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.050456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.076201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.077493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.077824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.078101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.078831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:49.079137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.080626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088bea0-7acf-4dc6-b51a-cbf8b90bc21d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.094941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 389/777] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:49.096290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9ef2-f73e-458c-b5a7-79993e8d9816 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.097393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9ef2-f73e-458c-b5a7-79993e8d9816 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.097393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf9ef2-f73e-458c-b5a7-79993e8d9816 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.098429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 390/778] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:49.098855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf9ef2-f73e-458c-b5a7-79993e8d9816 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.171294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.176571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e316d-48a8-4f03-b050-b16cf19bc7a6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.218251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.219199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.219555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.219838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.220616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:49.220915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.222229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf9ef2-f73e-458c-b5a7-79993e8d9816 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.222353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.247622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 389/779] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:49.247622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.263556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.264274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.264627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.264895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.265644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.267283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eaaf00-4b81-48c3-a531-81180c809872 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.280822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 391/780] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:49.282869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.283294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.283476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.303989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:49.305183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:49.339894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.365910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec962a1-6c6a-4b65-86d4-da06a0ec7ee5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.385275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 392/781] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:49.386159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.408066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.409207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.409765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.410292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.437490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.439005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:49.459428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:49.460807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:49.461942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:49.461942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:49.461942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:49.461942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 390/782] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:49.466145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.484458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.541547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f21072-44bc-49eb-9c72-67c6de4f08bd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.597130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.598689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.599144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.599568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.603865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361b9ce-03ed-49bb-aa3f-cb3d1fabc8e4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.614462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.616844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c6e41c1e-a727-4582-aaff-87a97da49efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:49.637083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:49.638267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:49.640072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:49.640072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:49.640072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:49.641694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 393/783] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:49.642529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.643206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.643486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.643804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.644144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebc44-c0d4-4ef7-96a4-495202eb5923 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.644466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:49.644816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.645016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebc44-c0d4-4ef7-96a4-495202eb5923 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.645458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ebc44-c0d4-4ef7-96a4-495202eb5923 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.646247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ba696-3de2-4bac-bb22-57efec4fd4ba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.647270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 394/784] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:49.648542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ebc44-c0d4-4ef7-96a4-495202eb5923 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.662934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 391/785] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:49.664240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.664525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.664887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.682761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.685559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:49.688453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:49.760297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.767792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 392/786] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:49.768624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.782806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ebc44-c0d4-4ef7-96a4-495202eb5923 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.831928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.833087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.833508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.833918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.835163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:49.835908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.837749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.855744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 395/787] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:49.857279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.894063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4698-70dc-4334-a9f8-9d6f449fc3e1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:49.933242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:49.933900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:49.934179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:49.934439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:49.943036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:49.944518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:49.949490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:49.959688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:49.960722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:49.961687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:49.965941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 393/788] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:49.966930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:49.975902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8313d3c-b3aa-46ff-b16a-0c66b9cb8423 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.011952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.012969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.013469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.013986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.025981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:50.031523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:50.032487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:50.033371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:50.034019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.036456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 396/789] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:50.037698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.082806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e963-f938-4cff-a25c-66edf19e7d97 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.119106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.119653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.119936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.120208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.127929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.129472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:50.141860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:50.142837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:50.143829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:50.147770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 394/790] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:50.148627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.155681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88d378-cea1-40b6-8430-d57cc511ee89 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.193121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.193740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.194019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.194293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.195026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:50.195332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.196777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.206931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.220841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 397/791] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:50.221701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.261182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e6ee1-159a-4abb-9309-da71961a5929 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.285226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.303104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.303755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.304077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.304257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.304967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:50.305325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.306629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.320927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 395/792] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:50.322235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.341986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57903d95-20c9-430d-bf06-b362c25d19fc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.377766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.378774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.379258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.379716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.380495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.381956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:50.392199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:50.393814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:50.395127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 398/793] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:50.396358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.448948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0898c3-345b-4780-b92c-e42a35b25e9c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.471085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.488605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.488812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.489096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.489366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.490063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:50.490369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.491858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.509955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 396/794] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:50.511086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.522895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251817e-ae9c-428b-a090-154c2cb4c045 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.558788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.560012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.560765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.561428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.561428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.573083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.574542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:50.590132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:50.591153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:50.592090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:50.592090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:50.592090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:50.592090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 399/795] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:50.592090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.633013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca0465-12f9-4d4f-8075-65fd036e74ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.669269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3153-9561-4a17-b508-67c04424513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.669889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3153-9561-4a17-b508-67c04424513e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.670172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3153-9561-4a17-b508-67c04424513e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.670631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3153-9561-4a17-b508-67c04424513e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.681604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d3153-9561-4a17-b508-67c04424513e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.683069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d3153-9561-4a17-b508-67c04424513e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-273d3153-9561-4a17-b508-67c04424513e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:50.699513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:50.700744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:50.702266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:50.702266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:50.702266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:50.702266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 397/796] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:50.702266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3153-9561-4a17-b508-67c04424513e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.710891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf1e89-36fc-4caa-a19e-1de30ad16958 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.725660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.748301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.749666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.749950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.750224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.750991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:50.751297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.752698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.757807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 400/797] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:50.762724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.808540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:50.819950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3153-9561-4a17-b508-67c04424513e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.860688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.861459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.861639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.861919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0af35a01c7c6427681a1f12508c1ff35/roles/f239192299174e29a4672c9df068e024` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.873393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d164b-71a2-495d-a301-70fdd2cf103f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:50.879436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0af35a01c7c6427681a1f12508c1ff35, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.881355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.898941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 398/798] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:50 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0af35a01c7c6427681a1f12508c1ff35/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:50.901571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:50.915738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:50.916692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:50.917120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:50.917566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:50.918391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:50.918847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:50.920657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:50.942342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 401/799] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:50.943665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.003678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.069810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7562bf-ddd9-4590-8a81-733ee1f09b3a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.111046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374830bf-d918-4e75-90ad-3b43690aa6b1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.115683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.116380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.116758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.117106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.117854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:51.118229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.119685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.137678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 399/800] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:51.137678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.142464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.173519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.174048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.174332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.174610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.175730 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:51.176039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.177695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.188809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 402/801] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 29 21:21:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:51.188809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.273321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ad8-7d65-437e-a1be-da008d6c19c5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.318388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.319196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.319598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.320018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.327689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b33090-c054-43e8-89db-79c5f06ba864 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.328490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.329301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.331197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:51.346945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:51.347990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:51.348934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:51.356158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 400/802] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:51.357435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.365660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.366298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.366574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.366844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.375031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.376467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:51.388656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:51.390186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 403/803] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:51.391161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.435324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.480709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176955db-bf84-4dc7-8c93-4e0891ddd6a5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.519538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.520322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.520732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.521110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.521786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.523116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5bd02-5736-43d7-8e57-854d393e323e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.523237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d472e0-d1d6-4dbc-88b6-ac8fceca59cf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.546308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 401/804] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:21:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:51.548098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7db0a8-3785-4e21-b9c8-2a461605aea3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.548652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7db0a8-3785-4e21-b9c8-2a461605aea3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.549133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7db0a8-3785-4e21-b9c8-2a461605aea3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.551052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 402/805] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:51.552169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7db0a8-3785-4e21-b9c8-2a461605aea3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.585370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.585981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.586361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.586707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.587694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:51.588104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.589720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f552b5-5cf1-4014-a25f-ffb2b9279440 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.610642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 404/806] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:51.610642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.610642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.610642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.636625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:51.640300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:51.659407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.719073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7db0a8-3785-4e21-b9c8-2a461605aea3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.726199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 405/807] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:51.726996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.754246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:51.775570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.776189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.776468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.776821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.777562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:51.777861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.779297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.802258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 403/808] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:51.809374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:51.854388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c141-662f-4132-b708-c66f1c0de2df None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:51.903678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:51.904357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:51.904707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:51.905567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:51.908066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0446b5664ae0453a8ccd57dfc562eb9b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:51.909525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:51.927493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 406/809] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:51 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:51.989615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0023f-10ae-4c45-ba77-389d3841c5c8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.043799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.044450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.044761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.045034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.058325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.059833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:52.076641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:52.077772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:52.079411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:52.079411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:52.079411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:52.079411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 404/810] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:52.085769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.099482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:52.196568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.217710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7920b37-1b9d-4abc-8089-548ad717b800 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.240935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:52.270311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.271038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.271323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.271596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.272246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:52.272549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.274617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.290979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 407/811] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:52.297004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.349743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35a45b-f471-4c84-8df3-9817ec702091 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.369840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:52.404854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.406952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.407452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.408176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] PATH_INFO: `/v3/services/network` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.409046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.409808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e002033-beea-4584-b5d1-e1e225904ea6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.410888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:52.423924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:52.425008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:21:52.425860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:52.426623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 405/812] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:52.437857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.517439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.518018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.518607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.518607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.538183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.539655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.565148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:52.570626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:52.571718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:52.572768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:52.581039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 408/813] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:52.582460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13a6fe-de43-47dd-b367-fc280b57294c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.587093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.630858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.633408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.634323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.636049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.636997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] RBAC: Adding query filter params (name=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:52.638832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.640561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.654942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 406/814] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:52.664237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.828803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bd7e8-baba-4375-b2d9-450fe368be93 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.829432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510b4c8-db1f-42ed-aba1-06c71b7e17e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:52.878558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.880910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.881638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.883241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.884137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] RBAC: Adding query filter params (type=network) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:52.884971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.887215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.887756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:52.903416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 407/815] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:52.910585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:52.911578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:52.912063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:52.912537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:52.913465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:52.913973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:52.915604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:52.927878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.949997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 409/816] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:52.955886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:52.978595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:53.152960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f3f3-da75-4841-ac28-6e410e168642 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:53.174191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663adff-1c19-4536-bd2a-45c5ad5e322c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:53.196675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:53.197345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.197627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.197891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/06c36d7a8a2246a38fe16e32a4607aae/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.200072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=06c36d7a8a2246a38fe16e32a4607aae, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:53.201666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fff418-7a5e-4938-a325-4d86493a191c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:53.215333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:53.216278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.216797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.217916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.219347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:53.221533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 410/817] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:21:52 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/06c36d7a8a2246a38fe16e32a4607aae/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:21:53.222127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:53.223162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c26c-b2a9-46de-bbe1-9cbc25be8cbb None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.223656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c26c-b2a9-46de-bbe1-9cbc25be8cbb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.223938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5c26c-b2a9-46de-bbe1-9cbc25be8cbb None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.230377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 411/818] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:53.237895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cec57-1a5f-41dc-be30-748908d16e84 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.238520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cec57-1a5f-41dc-be30-748908d16e84 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.238993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cec57-1a5f-41dc-be30-748908d16e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.267506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0a9cec57-1a5f-41dc-be30-748908d16e84 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:53.269344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a9cec57-1a5f-41dc-be30-748908d16e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:53.280724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 408/819] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:21:52 2022] POST /identity/v3/endpoints => generated 337 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:21:53.329574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:53.370764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 412/820] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:21:53.689079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:53.779016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:53.805076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004c40-b83e-4aaf-9489-16c096ddd8ad None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:53.842875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:53.843507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.843965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.844242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.855685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:53.910878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9274af-e0f3-4022-b1c6-9c81a3c34aae None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.911452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9274af-e0f3-4022-b1c6-9c81a3c34aae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.911452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9274af-e0f3-4022-b1c6-9c81a3c34aae None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.913400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 413/821] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:53.919696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:53.920054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:53.920274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:53.940641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:53.941837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:53.982061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:53.983952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:53.984906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 409/822] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:53.990007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.027523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 414/823] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:54.108638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9118b2-d65c-468e-bacc-2f0b2c38e290 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.147921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.148529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.148841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.149113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.149790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:54.150125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.151456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.153967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.161728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 410/824] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:54.166890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.274620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a2785-ea99-44b4-aadd-d38a6de6fcdf None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.276897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.315292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.316221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.316694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.317104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.320393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.329207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.331007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:54.347445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:54.348756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:54.349877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:54.349877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:54.349877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:54.351601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 415/825] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:54.357472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.431767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7985d-9714-4743-a327-54e4103d1da3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.465986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ff6f4-141e-4895-8b4e-36b6db0faaf4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.470481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.471235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.471450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.471724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.472307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.473698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cb62f48c-bdc6-4dcb-b914-1a6aa4ef14f1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:54.484950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:54.486146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:54.487008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 411/826] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:54.488317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76583fe9-f868-4993-b087-cffec373092c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.488317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76583fe9-f868-4993-b087-cffec373092c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.488472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76583fe9-f868-4993-b087-cffec373092c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.489869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 412/827] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:54.493529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76583fe9-f868-4993-b087-cffec373092c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.513728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.514776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.515063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.515323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.515964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:54.516261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.517653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dbc69-0306-4060-b736-6859ee71578b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.534977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 416/828] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:54.534977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.534977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.534977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.551073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:54.552247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:54.564172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.572614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.616088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76583fe9-f868-4993-b087-cffec373092c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.636545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 417/829] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:54.636545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.671983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.673949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.674786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.675287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.676421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:54.677226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.680108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.686767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 413/830] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:54.693519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.761084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0eea4-deeb-4ec7-80e4-ad93fcadeee0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.805266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.806278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.806755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.807235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.818622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.820920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.822585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df4b0c-97a2-4604-a03f-3746c59183b7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:54.834754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:54.835769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:21:54.836760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:54.839394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 418/831] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:54.849326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:54.873295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.873432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:54.877693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:54.878367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:54.878651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:54.878924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:54.879581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:54.879886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:54.881340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:54.888629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 414/832] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:54.914427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.016865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c4322-98d4-44b6-bb0a-d8fee25702b4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.058108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a44a6e-ea43-4e00-91c9-c0a5581d6982 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.064603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.065266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.065553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.065831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.066744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:55.067177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.069372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.090704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 419/833] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:55.096509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.102729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.103364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.103637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.103904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.118429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:55.124475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:55.125487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:55.126364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 415/834] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:55.131560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.173040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.196702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.218571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e7444-7d2b-414a-986a-7aa93bc673a8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.260578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db69541-7281-40fc-866d-be309146774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.261711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db69541-7281-40fc-866d-be309146774c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.262042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20479096-ed63-49f5-97e5-4ba08b8df19c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.262539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db69541-7281-40fc-866d-be309146774c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.263273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db69541-7281-40fc-866d-be309146774c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.312188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.312908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.313325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.313733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.314759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:55.315109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.317248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.332953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 416/835] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:55.335702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.351734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db69541-7281-40fc-866d-be309146774c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.353578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db69541-7281-40fc-866d-be309146774c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.437145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.460450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748c20-db64-420d-9c68-d311bf42a4e0 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.471648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 420/836] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:21:55.474493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.478156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db69541-7281-40fc-866d-be309146774c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.500217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.500889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.501168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.501436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.515121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.516617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fb4b776d-fddb-4e8f-89b5-30b74f666476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:55.531963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:55.533039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:55.533970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:55.533970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:55.533970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:55.533970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 417/837] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:55.534895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169e64-e6cc-42b1-bb21-a5766e00b6c2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.534895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169e64-e6cc-42b1-bb21-a5766e00b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.534895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169e64-e6cc-42b1-bb21-a5766e00b6c2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.536183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 418/838] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:55.540808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69169e64-e6cc-42b1-bb21-a5766e00b6c2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.592776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db69541-7281-40fc-866d-be309146774c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.628779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.629415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.629695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.629969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.643363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.644863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.656423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69169e64-e6cc-42b1-bb21-a5766e00b6c2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9ff321b0-38a7-44c8-88c6-c78dd74e170a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:55.660379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:55.661380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 421/839] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.667210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.684669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:55.685936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:55.698679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.716864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.718973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.718973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.718973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.719605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:55.719790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.721192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.738125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 419/840] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:55.738125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.764682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:55.767945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 422/841] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:55.770382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.860465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38fa04-0ea3-4e71-a4f4-22995d84f231 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.904211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.905035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.905410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.905869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.906873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:55.907269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.908984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.910828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5196c08-f688-40a5-bd7e-680c6e671332 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:55.925002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 420/842] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:55.930095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:55.968322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:55.969016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:55.969271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:55.969543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:55.980219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:55.981775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:55.994043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:55.995161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:55.996128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 423/843] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:56.002813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.019911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.057275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.079127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfff5-9dac-4e2b-86f7-7914745c641a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.116692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.117361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.117640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.117912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.125802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.127359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.138394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d389-f28a-43df-8067-0dc09f86bd2a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:56.139820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:56.140852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 421/844] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:56.141795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.178270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.178872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.179151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.179423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.180225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:56.180548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.181975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.204631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 424/845] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:56.204631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.266801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cb7dc-7c90-4235-a5ee-98ca26b80127 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.275701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.306561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.307188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.307465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.307735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.318497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbba6-5744-4f9b-a9ff-2c2afb3c9a8e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.319986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:56.325227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:56.326316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:21:56.327212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:56.331724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 422/846] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:56.332384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.336175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.358662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.359425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.359564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.359893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.360845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:56.360999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.362324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.377331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 425/847] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:56.378311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.454366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d672553-43a5-431e-a6fc-2a3a024dc9eb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.493022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.493260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9244b-82b3-4260-9d70-c91fae1d0e1a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.494116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.494704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.495179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.521943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.537954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.538678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.539031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.539369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.540130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:56.540501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.541969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.552639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 426/848] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:56.552816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.585324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.586826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.589920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.672959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c5314-d91a-46b0-bc78-ea634c2d2921 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.708566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 423/849] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:21:56.710225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.713907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.715406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.715684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.715954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.716706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0af35a01c7c6427681a1f12508c1ff35) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:56.717547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.718977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.736639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 427/850] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/role_assignments?user.id=0af35a01c7c6427681a1f12508c1ff35&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:56.736639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.796682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.860869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a029-fe24-4e7c-a8cf-a002c6d5368e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.877024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacac3a-ddb0-4276-80dc-b11fc565002a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:56.881436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:56.906725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.907366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.907647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.907924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.919778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.921845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.925958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:56.927221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.927221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.927466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-78919d6a-6d6c-4788-891c-047f298c2fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:56.939419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:56.940615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:56.942001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:56.942001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:56.942001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:56.942001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 428/851] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:56.942422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbbbd-eecb-43c4-9c2b-d9ef61d90e68 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.942722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbbbd-eecb-43c4-9c2b-d9ef61d90e68 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.943011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bbbbd-eecb-43c4-9c2b-d9ef61d90e68 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.945000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 429/852] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:56.945143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:56.946582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:56.949145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bbbbd-eecb-43c4-9c2b-d9ef61d90e68 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-f5eac646-68c7-46a6-b56e-7e6d12721a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:56.964679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:56.966620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 424/853] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:56.968791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:56.989474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:56.990698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:57.070731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 425/854] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:57.071819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.088635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bbbbd-eecb-43c4-9c2b-d9ef61d90e68 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.091109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.127846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.128458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.128794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.129064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.129709 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:57.130010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.131473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.147516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 430/855] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:57.155880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.201032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.217793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13b393-022a-44d0-b716-71dc2750d583 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.253275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.253900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.254189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.254471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.255126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:57.255496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.257100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.260295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa2700-c7ff-4cde-b778-fcf331b6cf88 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.270086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 426/856] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:57.277620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.298455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.299145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.299427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.299745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.309037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.310813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:57.322824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:57.323808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:57.324728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:21:57.326983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 431/857] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:21:57.333712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.361850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.407435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fad272-59a2-460d-8521-562f9f8621c3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.449503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.450141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.450419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.450692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.458500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32292-9b77-41e8-b926-4ee3fdfb78e9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.458720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.460073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:21:57.472502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:57.473711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 427/858] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:57.474814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.487297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.512052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.512964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.513251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.513529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.514316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:57.514835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.517197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.535397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 432/859] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:21:57.536293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.591230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e98ca1-c7c0-447e-a441-14a0fa9cc110 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.626886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.639346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.639959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.640242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.640557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.641249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.642629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.648963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfee7e-454f-453c-bd39-00b186744392 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.659583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 428/860] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:21:57 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:21:57.660692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.713826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.714741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.715265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.715738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.716748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:57.717771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.719599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.742884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 433/861] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:57.752672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:57.767876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:57.819887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8b1cc-afba-475d-a9f6-bb6fac235c26 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.868911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.869749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.870024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.870289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.871015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:57.871329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:57.873491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:57.880163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 429/862] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:57.898700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085f3a9-d8eb-453f-9fff-6bdef63909d5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:57.943411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:57.944043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:57.944319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:57.944606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:57.948766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:58.024852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:58.026354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:58.137724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 434/863] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:21:57 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:21:58.143648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:58.259289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80aa28-6ebb-4c7c-ace1-523e9ab5ca50 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:58.290900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:58.297785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:58.298471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.298748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.299156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.311678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:58.313164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:58.328848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:58.329879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:58.331159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:58.331159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:58.331159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:58.331159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 430/864] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:58.337791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:58.375688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:58.447815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e943f-189d-402b-8e8e-0901680deccc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:58.487183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:58.488103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.488560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.489032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.489873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:58.490352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:58.491886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:58.504776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 435/865] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:58.515146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:58.671890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43750-e2cc-4ee5-844d-424762df4e57 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:58.709624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:58.710357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.710641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.710914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.718827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:58.720293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-37e230e4-36e3-448a-a681-322e6b83b7e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:21:58.732012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:58.733030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:58.734325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 431/866] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:21:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:58.739424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:58.840853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956317-9b65-437b-8b64-15874f8db22a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.841180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956317-9b65-437b-8b64-15874f8db22a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.841402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae956317-9b65-437b-8b64-15874f8db22a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.843759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 432/867] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:58.848846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.849127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.849394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.852708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:58.864797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae79f7-aca6-40e9-974d-50d25f3dae2e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:58.871797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:58.874300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:58.906547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:58.907519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:58.908070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:58.908521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:58.909540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:58.910127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:58.911926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:58.927407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:58.932925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 436/868] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:58.939449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:58.980704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 433/869] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:21:59.072769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce50df2-f750-454f-bf36-b1e7c053a6f9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:59.114461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:59.115476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.115973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.116454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.117439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=06c36d7a8a2246a38fe16e32a4607aae) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:59.117954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:59.119595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:59.127507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 437/870] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:21:58 2022] GET /identity/v3/role_assignments?user.id=06c36d7a8a2246a38fe16e32a4607aae&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:59.278983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:59.394061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de8fca-4f2e-4fad-a8c7-5285b3e82322 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:59.432841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:59.433843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.434344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.434839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.447137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:21:59.452668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:59.453644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:21:59.454696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:59.457505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 434/871] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:21:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:59.464616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:59.487408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:59.571313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971287-a5ac-4e0c-b229-174b7a10b4b5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:59.610771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:59.611406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.611703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.611978 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.612688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:21:59.612991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:59.614373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f11-0381-4a11-9274-cdb1de541f26 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:59.623493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 438/872] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:21:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:21:59.629174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:59.664072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:59.740646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e1e5-cec2-476d-a049-104b9612296a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:59.775660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:59.776296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.776573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.776864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.787986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:59.789461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:59.803521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc59b-80d0-40b0-89f8-2358a868ffb7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.803823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc59b-80d0-40b0-89f8-2358a868ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.804048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc59b-80d0-40b0-89f8-2358a868ffb7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.805701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 439/873] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:21:59.807933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:21:59.809851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:21:59.810793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:21:59.810793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:21:59.810793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:21:59.810793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 435/874] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:21:59.814341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c14bf5-1e97-440e-9c7f-c6f1b47f2c28 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:59.814815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c14bf5-1e97-440e-9c7f-c6f1b47f2c28 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:59.815372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c14bf5-1e97-440e-9c7f-c6f1b47f2c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:59.817491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:59.838444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-10c14bf5-1e97-440e-9c7f-c6f1b47f2c28 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:59.839625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-10c14bf5-1e97-440e-9c7f-c6f1b47f2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:59.929958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 440/875] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:21:59.942558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:59.962354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f791fe-7f01-4b28-8f5a-56674352c988 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:59.999622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.000614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.001094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.001630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.002474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:00.002969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.004568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.018125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 436/876] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:00.022846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1293-c84c-4f18-aad2-f994d1b05298 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.022846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1293-c84c-4f18-aad2-f994d1b05298 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.022846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1293-c84c-4f18-aad2-f994d1b05298 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.024175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.026717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 441/877] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:00.028151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.028428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.028750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.051623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:00.052829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:00.063717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.140023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 442/878] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:00.149605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598c01-892e-46a4-befa-2ef0ab0ec097 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.192000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.193131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.193863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.194562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.209372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.210911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:00.224024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:00.225101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.226044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 437/879] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:00.230915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.236467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.241205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.354110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5123ab-fad4-4714-bc4c-6aa056dd9747 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.375877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00404f-71a4-4e3e-8241-44a4b57ae173 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.396211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.397221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.397746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.398155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.399160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:00.399744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.401400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.424718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.430727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 443/880] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:00.436699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.436878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.437350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.437687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.437954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.438565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.439938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:00.449153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:00.450167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.451051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 438/881] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:00.453526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.530984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.567494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb84716-f46c-4874-bdbc-632d05d5ebd8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.580178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f322e4-84fe-4d7d-9e13-d9c8e8306f94 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.611950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.612594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.612895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.613191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.624801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.625436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.626005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.626291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.626884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.628260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:00.637232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:00.638292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:00.639859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 439/882] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:00.640868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.687599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.690982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.692525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.749588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e83532-ded5-4a92-a5fc-617712a16fcf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.784120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.790350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.790964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.791237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.791503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.792249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:00.792547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.793936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.799325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 440/883] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:00.801579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.806562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 444/884] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:00.807405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.917602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084763eb-4352-443e-a2b9-94e11703fe07 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.934384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:00.940144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998486-6dd3-4f27-b564-8e1ac2d3d0a4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:00.958101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.958820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.959105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.959743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.960121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:00.960524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.961844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.973971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 441/885] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:00.973971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:00.989298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:00.989902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:00.990183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:00.990458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:00.991088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:00.991385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:00.992779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:00.997945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 445/886] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:00.999467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.042848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.089771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba4e3-eef3-4eb2-a0bf-60bbcae5e978 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.118989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39628b-4b32-43ef-90d2-57aa2dba87b3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.129018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.129966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.130574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.130854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.143632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.145325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.156559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.157185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.157460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.157727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.158354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:01.158645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.160210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:01.162581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:01.163764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:01.164721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:01.164721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:01.164721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:01.164721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 442/887] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:01.164964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.174290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 446/888] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:01.175133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.207517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.297274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b8ee3-d2ee-4470-8574-2939c3a986ab None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.298462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85304d6-1f3a-4c96-b6bc-38de3b7379aa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.314093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.341245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.342093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.342552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.342789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.343893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] RBAC: Adding query filter params (interface=public, service_id=2b8e81ac67da4088bfdab0477fbe13fe, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:01.343893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.344504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.345217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.345381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3820bb-beeb-42e3-8c22-35df7f290a99 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.345478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.345698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.346348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:01.347124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.348332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd73e61-7ff0-449c-90be-c69c13e2da93 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.351621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 443/889] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/endpoints?service_id=2b8e81ac67da4088bfdab0477fbe13fe&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:01.353010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716883e-329b-4094-a86f-c04a54cd1eed None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.353434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716883e-329b-4094-a86f-c04a54cd1eed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.353819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716883e-329b-4094-a86f-c04a54cd1eed None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.355517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 444/890] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:01.364680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716883e-329b-4094-a86f-c04a54cd1eed None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.369205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 447/891] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:01.370820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.371176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.371524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.391487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:01.392786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:01.472855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 448/892] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:01.473621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.491869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716883e-329b-4094-a86f-c04a54cd1eed None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.521248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.535578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.536702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.537505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.538209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.539063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.541576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.549657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 445/893] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/services => generated 1011 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:22:01.630230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36dd18-0f56-43fc-a082-e4f43ef5bcf6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.672185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.672937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.673217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.673488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.681555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.683098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:01.697370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:01.698336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:01.699251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 449/894] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:01.704736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.766844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.778062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.836822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f9d07-7168-4694-9cba-d6647061747c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.874634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.876059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.876442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.876842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.877652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:01.878063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.879642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.887513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:01.895796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 446/895] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:01.907657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:01.928130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f808b1-7f5d-44a8-9607-a455eea33742 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:01.965686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:01.966382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:01.966667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:01.967585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:01.968243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:01.969939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:01.979600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:01.980698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:01.981571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 450/896] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:01.989877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:02.030877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:02.034848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94008bcc-8c4c-4b88-8144-e0d4ca95c710 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:02.084663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:02.084663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:02.084663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:02.084663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:02.085797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] RBAC: Adding query filter params (role.id=b8f1915970ff43288cf21f586082f269, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0af35a01c7c6427681a1f12508c1ff35) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:02.086246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:02.087842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:02.094868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 447/897] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/role_assignments?user.id=0af35a01c7c6427681a1f12508c1ff35&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=b8f1915970ff43288cf21f586082f269 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:02.134698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a483f-99f5-4204-b68c-bd7e7b8fb1af None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:02.183818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:02.184456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:02.184811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:02.185156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:02.186069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:02.186575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:02.190267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26594fb9-7da0-4c2e-8bc0-4754ae3330ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:02.196881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 451/898] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:02.202317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:02.325260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a7bf2-c32d-4014-b8f7-721626950846 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:02.363119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:02.363807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:02.364092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:02.364363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:02.365055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:02.365360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:02.366792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58502f8-a519-4519-b64a-b0ac05e2ee6d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:02.372389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 448/899] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:03.239609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2d19-38a3-4036-9d1f-03c5011a522d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:03.240117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2d19-38a3-4036-9d1f-03c5011a522d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:03.240117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2d19-38a3-4036-9d1f-03c5011a522d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:03.241729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 452/900] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:03.246958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7e91-af3a-4ce2-ac00-60a57e93cf7c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:03.247484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7e91-af3a-4ce2-ac00-60a57e93cf7c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:03.247645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7e91-af3a-4ce2-ac00-60a57e93cf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:03.266570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5e3c7e91-af3a-4ce2-ac00-60a57e93cf7c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:03.267768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5e3c7e91-af3a-4ce2-ac00-60a57e93cf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:03.292003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:03.344708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 449/901] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:03.613422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748f9d-e479-4e19-af6e-0090589e7e6b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:03.613901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748f9d-e479-4e19-af6e-0090589e7e6b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:03.614280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c748f9d-e479-4e19-af6e-0090589e7e6b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:03.616040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 453/902] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:03.622056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:03.622735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:03.623841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:03.627233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c748f9d-e479-4e19-af6e-0090589e7e6b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:03.637556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:03.643817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:03.645097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:03.717327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 450/903] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:03.746153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c748f9d-e479-4e19-af6e-0090589e7e6b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:03.784593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:03.785226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:03.785508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:03.785780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:03.786514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:03.787945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3a183-3948-4422-85d5-da48bc3e39a1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:03.812946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 454/904] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:03 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:22:03.995581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.102253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:04.118013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f882f-fc34-43e2-b731-254ec07a4ce8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.155867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e2b8-2604-4cb9-bc82-3e0d3543146f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.156440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e2b8-2604-4cb9-bc82-3e0d3543146f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.156694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e2b8-2604-4cb9-bc82-3e0d3543146f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.158175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 455/905] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:04.163307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540511-efad-41dc-af66-3b383d908be0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.163592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540511-efad-41dc-af66-3b383d908be0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.163971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72540511-efad-41dc-af66-3b383d908be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.172255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.173403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.173860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.174322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.175076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:04.176634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:04.183770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-72540511-efad-41dc-af66-3b383d908be0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:04.185135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-72540511-efad-41dc-af66-3b383d908be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:04.187501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:04.188491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:22:04.189385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:04.192179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 451/906] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:04.198395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.260317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:04.272507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 456/907] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:04.317355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63f0bd-f379-4747-8c06-be4e3dbb2c7b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.353748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.354666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.355151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.355596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.356403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:04.356911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:04.358553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451f132-a19f-4ccf-aab8-a01ba660b9a9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:04.364203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 452/908] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:04.370137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540511-efad-41dc-af66-3b383d908be0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.438192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0859c-fa2f-42c1-a0c6-f466e1cbbda3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.439196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0859c-fa2f-42c1-a0c6-f466e1cbbda3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.439932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0859c-fa2f-42c1-a0c6-f466e1cbbda3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.443261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 453/909] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:04.453231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.454156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.454892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.480762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:04.482272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:04.492718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:04.505208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72540511-efad-41dc-af66-3b383d908be0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.551909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.552912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.553195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.553466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.554111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:04.554464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:04.555866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:04.561693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 457/910] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 29 21:22:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:04.562533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.579327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 454/911] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:04.580536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.652313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:04.691174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80dd62-9ed7-4d82-a4f1-37eac5dfc7ae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.708795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ead4c2-e254-45c3-b090-8e7691ab669e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.730247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.730965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.731319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.731753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.732946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:04.734397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:04.751430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.751624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:04.752796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.753331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.753882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.758239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 455/912] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:04.767675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:04.773465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:04.774526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:04.775390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:04.778638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 458/913] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:22:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:04.785459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.871133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.915653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233980be-9bee-4a4d-899a-72642e45075f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:04.953994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:04.954726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:04.955009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:04.955395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:04.956080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:04.956383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:04.957800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:04.967826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 456/914] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:22:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:04.974837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:04.995560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67565c73-ba0a-4015-ad9f-e86184350136 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.016888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:05.034390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.035023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.035305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.035580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.036250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.037654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.049326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:05.055102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 459/915] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:04 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:22:05.103570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ec828-6858-43aa-990a-5f13dda963e1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.153886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.155206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.155664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.156202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.170639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.172260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:05.189027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:05.190048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:05.190991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:05.190991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:05.190991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:05.190991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 457/916] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:05.196087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:05.320268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4629e-847f-40aa-add4-27f6c90f5a09 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.358583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.359586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.360085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.360553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.361443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:05.361953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.363537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.376973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 460/917] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:05.384634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:05.497043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224537c-54f4-4910-9b7f-354f7109ec50 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.499430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:05.533734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.534377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.534659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.534931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.542924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.544382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:05.556240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:05.557312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:05.558305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 458/918] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:22:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:05.563184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:05.608416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:05.673042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ac6fc-828a-4da2-8499-cbb634472c8f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.710217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.710830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.711112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.711387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.712116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:05.712422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.713880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ff97a-d213-4a3e-9444-2112a715122d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.732491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 461/919] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:22:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:05.738037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:05.845172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9c075-deff-40cc-b33b-1ea5f1503e63 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:05.878163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd5ed-95ad-4c3b-912d-3c24352a401b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.878697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd5ed-95ad-4c3b-912d-3c24352a401b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.879186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd5ed-95ad-4c3b-912d-3c24352a401b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.881266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 462/920] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:05.886699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.886986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.887262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.893354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:05.894496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:05.897015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:05.897015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:05.910409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:05.912229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:05.912963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:05.914215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:05.932209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:05.933291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:05.934237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:05.934237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:05.934237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:05.941255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 459/921] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:05.942313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:05.962899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:05.996680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 463/922] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:06.053298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42a029-2d68-401c-b87e-aecf5087b1a2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.088897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.089811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.090285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.090754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.091576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:06.092106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.093761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.105931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 460/923] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:06.115518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.146183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.229901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229dbb9-4f40-4afa-8703-6f6cbe33318d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.267339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.268354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.268887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.269385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.277736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.279612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.292372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:06.293105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:06.293984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:06.295437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:06.298079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 464/924] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:06.303478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.399946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.404854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.421012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab5d06-bf5a-4c63-80ee-8683dd960443 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.434504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead504d-ee2d-4f5c-b917-7346960a08e1 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.475789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.476283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.476609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.476976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.477323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.477689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.478083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.478345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.478790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.479660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.479767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:06.480242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.482169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:06.491320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:06.492311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:06.493553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:06.497138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 461/925] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:06.498508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.500847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 465/926] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:06.507807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.618064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6486bc-c7f4-4623-9df0-cb96a85b6b72 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.618523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57173b-3643-42a7-8d62-e6282b9cdf89 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.654891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.655885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.656645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.656770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.657248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.657431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.657684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.657999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0af35a01c7c6427681a1f12508c1ff35/roles/b8f1915970ff43288cf21f586082f269` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.658663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:06.659175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.660239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0af35a01c7c6427681a1f12508c1ff35, role_id=b8f1915970ff43288cf21f586082f269)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.660976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.661700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b894ffc-438d-4856-b48a-27d97541b765 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.667066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 462/927] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:06.674164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.678627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.687166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 466/928] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:22:06 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0af35a01c7c6427681a1f12508c1ff35/roles/b8f1915970ff43288cf21f586082f269 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:22:06.792408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d7ffe-779b-4d3f-8311-c400f3dc7d8b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:06.829874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48792ac-53dd-492e-b38d-62c86fe46258 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.830238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48792ac-53dd-492e-b38d-62c86fe46258 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.830429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48792ac-53dd-492e-b38d-62c86fe46258 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.831924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 467/929] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:06.834916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:06.835669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.835878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.836155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.836827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:06.837134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:06.837716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:06.838163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:06.838445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:06.838552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:06.845097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 463/930] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:06.851983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:06.869080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:06.870533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:06.873084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.936522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:06.957800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 468/931] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:06.973640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7d6c-3a47-4a31-b82f-278ae96d876d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.009594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.010204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.010483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.010748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.011506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] RBAC: Adding query filter params (interface=public, service_id=116044654bfe4c0ca3b1c7128f42c3d2, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:07.011806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.013306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed975b2-aa5e-4fae-a1a8-2ef4088f5e67 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.019033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 464/932] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:22:06 2022] GET /identity/v3/endpoints?service_id=116044654bfe4c0ca3b1c7128f42c3d2&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:07.024279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.134260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27fa31-9004-4859-bbf5-9d47f2de2a40 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.147841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dd09f-5156-4491-97ce-63eafa08ed12 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.148167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dd09f-5156-4491-97ce-63eafa08ed12 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.148506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dd09f-5156-4491-97ce-63eafa08ed12 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.150273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 465/933] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:07.157716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.158823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.159827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.175047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.175661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.175938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.176209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.177277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.178916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.183511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:07.185007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:07.185494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 469/934] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:22:07.239480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.242599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:07.272763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 466/935] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:07.363630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7ada3-6659-4ea0-a4aa-37229b330e15 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.401243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.402642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.403142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.403613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.405517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.407502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.410344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING keystone.server.flask.common [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 21:22:07.431056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.sql.core [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:22:07.431056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 21:22:07.431056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [parameters: {'id': '1fbeddfaa9d24a5c9cff06c546a901f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 21:22:07.431056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81537) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fbeddfaa9d24a5c9cff06c546a901f6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fbeddfaa9d24a5c9cff06c546a901f6. Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:22:07.439523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 21:22:07.440327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application yield resource Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise self.value Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application session.commit() Aug 29 21:22:07.441378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:22:07.442307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:22:07.443884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application result.read() Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:22:07.444725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [parameters: {'id': '1fbeddfaa9d24a5c9cff06c546a901f6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.445624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:07.446431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1fbeddfaa9d24a5c9cff06c546a901f6. Aug 29 21:22:07.448515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.451413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 470/936] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 29 21:22:07 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:07.457166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.527865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:07.546805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.568287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1380df-a0a2-48c5-a911-84e69c61604d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.604260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.604915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.605191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.605458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.612917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.614370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.615000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:07.626384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:07.627549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:07.628550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 467/937] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:07.634966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.669317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba315a-0b05-4cc7-afa0-05c6b3d06c2b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.720885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.722013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.722300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.722615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.723333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.725082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:07.737257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:07.738356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:07.739236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 471/938] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:07.745772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:07.775521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae02799-8b3f-4c33-8960-6c254d723104 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.800649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:07.814168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.815099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.815713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.816191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.817121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:07.817681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.819536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.842754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 468/939] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:07.876628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035ccd7-69e6-45a1-8ef2-0479e47a2ba3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:07.908945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:07.922872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:07.923454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:07.923739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:07.924015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:07.924711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:07.925022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:07.926536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:07.935458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 472/940] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:07.941129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:08.049546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7102-ea28-4c2a-b4ca-42226ff0824a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:08.086272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:08.086809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:08.087092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:08.087368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:08.088151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:08.088728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:08.090305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:08.096346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 469/941] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:08.102358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:08.221833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7bf48-049b-4505-85bf-a856637dee19 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:08.261638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:08.262258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:08.262538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:08.262810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:08.263590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] RBAC: Adding query filter params (interface=public, service_id=76b4d168919848b484b1ff00c6ca67a7, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:08.264560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:08.266398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd409e-2a6b-4b72-81ad-81f24fade1a3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:08.274954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 473/942] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:22:08 2022] GET /identity/v3/endpoints?service_id=76b4d168919848b484b1ff00c6ca67a7&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:08.280686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:08.334944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:08.389724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c45a-93d3-4fcf-ad8b-957992a6de6a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:08.427117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:08.427728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:08.428006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:08.428278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:08.428998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:08.430366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac46356-686e-4307-804d-9c67825673b5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:08.437214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 470/943] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:08 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:22:08.804532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241559d5-34a2-45bc-9927-71eea7c36f85 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:08.805088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241559d5-34a2-45bc-9927-71eea7c36f85 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:08.805088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241559d5-34a2-45bc-9927-71eea7c36f85 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:08.806874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 474/944] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:08.813212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19ce7-b62c-4902-aa1c-bbedcbc26a62 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:08.813642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19ce7-b62c-4902-aa1c-bbedcbc26a62 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:08.813822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19ce7-b62c-4902-aa1c-bbedcbc26a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:08.832791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bdb19ce7-b62c-4902-aa1c-bbedcbc26a62 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:08.833989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bdb19ce7-b62c-4902-aa1c-bbedcbc26a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:08.914393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 471/945] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:09.190593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7b03-3bee-4853-88a0-9f4c3f9651f9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.190888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7b03-3bee-4853-88a0-9f4c3f9651f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.191072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e7b03-3bee-4853-88a0-9f4c3f9651f9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.192700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 475/946] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:09.198378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.198666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.198944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.206003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e7b03-3bee-4853-88a0-9f4c3f9651f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.219921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:09.221525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:09.264341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:09.267526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:09.300416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 472/947] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:09.331043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e7b03-3bee-4853-88a0-9f4c3f9651f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.367227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:09.367860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.368136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.368408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.380278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:09.385191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:09.386134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:09.387462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 476/948] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:09.394415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.499992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a41e7-8e91-40e4-ba83-a6ba72668fd0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.536533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:09.537172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.537451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.537725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.538457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:09.538763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:09.540139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:09.549524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 473/949] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:09.560796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.612839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.658567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:09.678495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96e64b-3b1a-47fc-b86f-3fe64631c77c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.690543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:09.715773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:09.716400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.717415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.717697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.729065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:09.730524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:09.747321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:09.748469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:09.749683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:09.749683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:09.749683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:09.752933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 477/950] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:09.754562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.761605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5e39a-6e51-4cad-9b22-5e934ab0fe72 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.809471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:09.810574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.811860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.812363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.813568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:09.816705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:09.832314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:09.833531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:22:09.834547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:09.843983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 474/951] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:09.846211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.886825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556f463-c787-4cf2-b1d8-57e85335a051 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.942805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:09.943408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.943687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.943945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.944609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:09.944933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:09.946303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12359614-2bd0-43a4-8e4f-fe6b7b3843be None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:09.959654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 478/952] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:09.965409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744d59-fef6-40e1-bca6-0ab8cbf4711b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:09.965600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744d59-fef6-40e1-bca6-0ab8cbf4711b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:09.965820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46744d59-fef6-40e1-bca6-0ab8cbf4711b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:09.968308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 479/953] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:09.969093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46744d59-fef6-40e1-bca6-0ab8cbf4711b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:09.985856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:09.993243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315a806-087b-4449-9c59-0c16988a8a39 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:09.995621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.034539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.035152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.035426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.035695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.036330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:10.036664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.038046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b5475-0193-4313-a42a-c91fe88022a6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.043104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 475/954] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:10.044318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.044636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.044912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.065200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:10.066405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:10.103187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46744d59-fef6-40e1-bca6-0ab8cbf4711b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.144653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 476/955] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:10.144653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.147591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.147591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.147591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.147591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.163986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.165818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:10.180817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:10.181887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:22:10.182888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:10.187330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 480/956] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:22:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:10.188775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.248079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.258858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5c2f-83bd-4ba2-a78e-45db7b879140 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.281971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.299067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.299710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.299994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.300263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.300923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:10.301223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.302708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.307201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b12dc7-3e3b-4e6f-93ab-a4648591778d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.312509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 477/957] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:10.314290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.353316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.354985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.355480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.356223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.358580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:10.359129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.361448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.390622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 481/958] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:10.392936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.442613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5f334-6bc3-4637-83fc-2cf1542cd89c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.488502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.489305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.489586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.489914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.490595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.491973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.524614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 478/959] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:22:10.524614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.524926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d392d-d99d-463d-b3e4-91a185cc4e2b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.527724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.580279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.580946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.581222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.581491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.585843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.641571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc81102-97f7-4cae-9b50-49643cd1237d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.664328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.665818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433a5d7-7ddb-4f4d-aab2-b9420d2452bb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.691057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.691672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.691947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.692222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.693018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.694467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.697034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING keystone.server.flask.common [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 21:22:10.713223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.sql.core [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:22:10.713223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 21:22:10.713223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [parameters: {'user_id': '60424945a9f244029c48e46672c04bfb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 21:22:10.713223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81536) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9ec61761-b49f-402a-b512-4c5928de561b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 29 21:22:10.732334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:22:10.733357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application yield resource Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise self.value Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application session.commit() Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:22:10.734352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:22:10.735271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:22:10.736208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application result.read() Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:22:10.737147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [parameters: {'user_id': '60424945a9f244029c48e46672c04bfb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:10.739865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 21:22:10.746427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 479/960] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 29 21:22:10 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fce2d-1a71-4f25-a20d-06b70cad1c39 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fce2d-1a71-4f25-a20d-06b70cad1c39 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.748754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fce2d-1a71-4f25-a20d-06b70cad1c39 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.750070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 480/961] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:10.750070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fce2d-1a71-4f25-a20d-06b70cad1c39 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.791729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 482/962] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:10.797681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.798189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.798318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.816428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:10.817971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:10.831764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.850059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:10.871418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fce2d-1a71-4f25-a20d-06b70cad1c39 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:10.901741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 483/963] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:10.902600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:10.919601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:10.920312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:10.920537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:10.920813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:10.928848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:10.930432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:10.942666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:10.943709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:22:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:10.948530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 481/964] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:10.949538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.040263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb0f3b-1b31-4f05-971c-f98d2e92ebb0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.077419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.077994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac1a1-2af9-47a0-b482-e24e9689083b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.078226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.078438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.078713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.094750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.097064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.111128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.118855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.119472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.119797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.120071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:11.120489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:11.121449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:11.122370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:11.122370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:11.122370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:11.122552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.122552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.122552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.125543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 484/965] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:11.137727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.154354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.174157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 482/966] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 29 21:22:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:11.179311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.253250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1198c-093a-4abf-9dbd-debae00cfdd4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.292766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.293262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.293538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.293808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.294450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.295343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.296740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.316370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 485/967] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:11.317563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.322684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7eaab-5683-4f5a-810e-9c0706e6f566 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.364416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.365053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.365328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.365589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.366261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.367720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:11.378135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:11.379291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:11.385598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 483/968] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:11.387110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.420177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.447759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443098f2-7ff1-43f8-b095-0c101f784a62 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.465220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.491156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.491795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.492075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.492349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.500214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.501822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:11.514086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:11.515356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:11.516499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:11.520633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 486/969] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:11.521188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f978fec-4a2f-448d-a3c3-8f0c4580cc27 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.522645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.558581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.559435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.559870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.560143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.561350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.561646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.563086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.572892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 484/970] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:11.574042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.646307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c629-a6e7-4e3c-81bf-380c1a079462 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.688391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119276-4160-4c76-b356-a51628787273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.689157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119276-4160-4c76-b356-a51628787273 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.689528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119276-4160-4c76-b356-a51628787273 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.690046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a119276-4160-4c76-b356-a51628787273 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.690934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a119276-4160-4c76-b356-a51628787273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.691636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a119276-4160-4c76-b356-a51628787273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.693993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a119276-4160-4c76-b356-a51628787273 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.703292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.710022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 487/971] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:11.719182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119276-4160-4c76-b356-a51628787273 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.719338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625c431-ed40-43b0-b7fe-cfa3a6bd7456 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.737584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:11.757356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.757971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.758249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.758524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.759169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.759470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.760901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.768201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 485/972] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:11.775194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:11.851458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a119276-4160-4c76-b356-a51628787273 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.893098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.893098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.893098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.893098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.894127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] RBAC: Adding query filter params (role.id=b8f1915970ff43288cf21f586082f269, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0af35a01c7c6427681a1f12508c1ff35) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:11.894546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.898269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ecb6c-4e14-4411-ac8a-049943d47d1c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.899887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65886f-2069-4ed2-8149-80dd6e8c1fbf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:11.905435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 488/973] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:11 2022] GET /identity/v3/role_assignments?user.id=0af35a01c7c6427681a1f12508c1ff35&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=b8f1915970ff43288cf21f586082f269 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:11.949605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:11.950248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:11.950531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:11.950836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:11.952074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:11.956792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1f618-32f0-4c24-a2ad-047a3004e4c9 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:11.996825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 486/974] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:12.537546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6e08-b936-4e35-863c-88578f13cf4f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:12.537862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6e08-b936-4e35-863c-88578f13cf4f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:12.538139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f6e08-b936-4e35-863c-88578f13cf4f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:12.539514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 489/975] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:12.545359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:12.545732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:12.546008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:12.564697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:12.566211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:12.598631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:12.653589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 487/976] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:12.935194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f6e08-b936-4e35-863c-88578f13cf4f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:13.001681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:13.044243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f6e08-b936-4e35-863c-88578f13cf4f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:13.082092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:13.082727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.083008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.083282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.083920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:13.085329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:13.094901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:13.095928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:13.096834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:13.098354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 490/977] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:13.101822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:13.207064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33127ba-5b94-428a-9594-15d79f9beac9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:13.240446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:13.241080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.241362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.241635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.242261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:13.242563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:13.243928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696b290-b6c3-4ca0-bf53-889263daa613 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:13.250598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 488/978] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:13.255789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:13.360878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb563c-e00a-4b06-a287-020a00365e0e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:13.386800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:13.397561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:13.398175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.398455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.398725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.399354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:13.399657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:13.401051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8aff-8f7a-44e2-bb44-469e0110ef45 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:13.406725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 491/979] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:13.901242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd7cc1-b944-47d4-802e-607fdce4de2d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.901567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd7cc1-b944-47d4-802e-607fdce4de2d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.901800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd7cc1-b944-47d4-802e-607fdce4de2d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.903298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 489/980] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:13.909236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.909529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.909861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.929161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:13.930401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:13.960037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f9802-53dc-4e04-b7e6-34b0251fa662 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.960365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f9802-53dc-4e04-b7e6-34b0251fa662 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.960608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f9802-53dc-4e04-b7e6-34b0251fa662 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.962072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 490/981] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:13.967376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:13.967676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:13.967956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:13.986500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:13.987683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:14.008632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 492/982] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:14.018893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:14.069550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 491/983] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:14.275097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.339890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.378029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76fe74-ebc9-4f32-b98f-03f38979e3e3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.409968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:14.426663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.427299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.427576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.427845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.428424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.429837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:14.440157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:14.441410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:14.442431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 493/984] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:14.446997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.459768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395dddd-0b2d-4703-b01a-f1db31652846 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.496794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.497437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.497795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.498568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.499160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.500550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:14.510149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:14.516688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:14.517570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 492/985] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:14.518387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.545727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:14.564649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627e6ab-a7ed-4f23-8fb9-ab533cd237e6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.607756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.608698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.609198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.609544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.609966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:14.610554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.611917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.620738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 494/986] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:14.625158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.632331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895bfbc-9381-47cf-b9d1-ac07d3f4ea7b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.678651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.679253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.679531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.679798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.680425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:14.680755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.682115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.683482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:14.687040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 493/987] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:14.693113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.742083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bfb71-d475-49a4-86f3-3bfb8e289c8b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.778533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.779163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.779444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.779716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.780338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] RBAC: Adding query filter params (type=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:14.780661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.782275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.786754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 495/988] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:14.791404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:14.810952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de3a69-7aee-46bb-94a7-8ce6173ba899 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.850423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:14.851061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:14.851340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:14.851612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:14.852260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:14.852559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:14.853972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:14.867780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 494/989] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:14.869021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:14.985957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fdeaf-5f5f-48f5-b8aa-665206d7ccab None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:14.999981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:15.022901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:15.023575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:15.023816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:15.024047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:15.024783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] RBAC: Adding query filter params (interface=admin, service_id=76b4d168919848b484b1ff00c6ca67a7, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:15.025094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:15.026457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798f030-7066-4ff6-8366-91dabb6fe721 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:15.036763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 496/990] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Aug 29 21:22:14 2022] GET /identity/v3/endpoints?service_id=76b4d168919848b484b1ff00c6ca67a7&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:15.037973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:15.142070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdacbc-5101-4541-a2d0-93acf703808d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:15.176446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:15.177083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:15.177361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:15.177632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:15.178285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:15.179700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e8df4-8387-4f19-aa6f-974157809135 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:15.192782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 495/991] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:15 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:22:15.352607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9c40-55d6-4f86-8fca-ab25012f6220 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:15.352997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9c40-55d6-4f86-8fca-ab25012f6220 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:15.353176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9c40-55d6-4f86-8fca-ab25012f6220 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:15.354779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 497/992] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:15.360003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172d991-589d-4767-92c0-87a63b6acd16 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:15.360443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172d991-589d-4767-92c0-87a63b6acd16 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:15.360634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172d991-589d-4767-92c0-87a63b6acd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:15.379514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4172d991-589d-4767-92c0-87a63b6acd16 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:15.380725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4172d991-589d-4767-92c0-87a63b6acd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:15.390905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:15.452535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 496/993] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:15.722304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d9c40-55d6-4f86-8fca-ab25012f6220 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:15.830242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d9c40-55d6-4f86-8fca-ab25012f6220 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:15.870207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:15.870843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:15.871125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:15.871401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:15.872090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:15.874304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00837d-8cc6-4b4f-9764-7b399e2b8d6a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:15.896801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 498/994] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:15 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:16.707217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0de3-c98b-41ad-bb58-c6b582630e4e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:16.707741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0de3-c98b-41ad-bb58-c6b582630e4e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:16.707741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c0de3-c98b-41ad-bb58-c6b582630e4e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:16.709297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 497/995] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:16.714263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c0da-8913-436f-b04c-1db38fe77ac9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:16.714656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c0da-8913-436f-b04c-1db38fe77ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:16.714816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c0da-8913-436f-b04c-1db38fe77ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:16.733656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6fc4c0da-8913-436f-b04c-1db38fe77ac9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:16.734829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6fc4c0da-8913-436f-b04c-1db38fe77ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:16.765272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:16.814452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 499/996] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:17.078796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c0de3-c98b-41ad-bb58-c6b582630e4e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:17.107025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a24-e6a7-4d4c-8847-87a0cca595ce None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.107362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a24-e6a7-4d4c-8847-87a0cca595ce None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.107595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a24-e6a7-4d4c-8847-87a0cca595ce None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.108974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 500/997] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:17.113668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.113936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.114217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.133366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:17.134542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:17.193630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c0de3-c98b-41ad-bb58-c6b582630e4e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:17.205686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 501/998] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:17.232894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:17.233634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.233821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.234067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.234731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:17.236101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:17.256638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 498/999] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:17 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:17.471543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:17.565110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:17.575045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc356ffc-bbf9-416c-88ce-3578c367af23 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:17.616004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:17.616656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.616937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.617207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.617780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:17.619134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:17.628359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:17.629483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:22:17.630942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:17.631944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 502/1000] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 29 21:22:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:17.635481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:17.639540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:17.749413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750384d0-4342-4b9b-81fe-ade175b8b04f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:17.787570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:17.788180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.788455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.788820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.789365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:17.789660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:17.791021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb984c30-d6b0-4cf4-af71-719902572898 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:17.796744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 499/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:17.802267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:17.852179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c762-cbaf-4157-b6cc-670da4702f0c None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.852493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c762-cbaf-4157-b6cc-670da4702f0c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.852777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7c762-cbaf-4157-b6cc-670da4702f0c None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.854388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 500/1002] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:17.860482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.860482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.861246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.880156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:17.881395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:17.917739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfc08b-13e2-4c96-8bf3-e42b16d02806 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:17.948931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:17.956631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 501/1003] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:17.959284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:17.959900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:17.960186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:17.960461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:17.961121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:17.961426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:17.962786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:17.968962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 503/1004] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 29 21:22:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:17.974298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:18.075668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14dfb9-c967-4a0d-86fb-530f62e021db None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:18.109641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:18.110263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:18.110612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:18.110953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:18.111745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:18.113302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:18.136622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 502/1005] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:18.236049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:18.257787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:18.348747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e9db7-c227-40fa-97db-c660fb7568ba None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:18.391520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:18.392156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:18.392435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:18.392737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:18.393308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:18.394668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:18.403477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:18.405133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:18.406311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 504/1006] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:18.413015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:18.473783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:18.517723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68fe10-7701-4822-b5a5-f32a579bb563 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:18.556434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:18.557064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:18.557341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:18.557609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:18.558229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:18.558527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:18.559884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:18.570591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 503/1007] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:18.571485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:18.660173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:18.686041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbdf0f-bf57-478e-8d59-321eb7470fd2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:18.720836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:18.721444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:18.721720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:18.722013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:18.722609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:18.722909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:18.724337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:18.732849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 505/1008] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:18.735369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:18.842371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fe50b-0ee2-4526-9fef-efbd5ea7c3eb None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:18.869943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:18.883506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:18.884120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:18.884401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:18.884707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:18.885416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] RBAC: Adding query filter params (interface=public, service_id=0a7745ee77f14797b3c5419642e2ec40, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:18.885758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:18.887239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3155a2-6681-409d-b58d-f23d4c190905 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:18.900736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 504/1009] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:22:18 2022] GET /identity/v3/endpoints?service_id=0a7745ee77f14797b3c5419642e2ec40&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:18.903599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:19.006564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0abe0-79bc-4fe3-ac09-6149b02bbfd3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:19.045616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:19.046226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:19.046504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:19.046773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:19.047377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:19.048865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb92581-3839-4919-9699-3d2443b4aa24 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:19.060968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 506/1010] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:18 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:22:19.195384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34293ae-027d-4916-881c-c2f92d1114af None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:19.195770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34293ae-027d-4916-881c-c2f92d1114af None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:19.196153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34293ae-027d-4916-881c-c2f92d1114af None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:19.197640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 505/1011] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:19.203508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:19.203798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:19.204078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:19.212333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:19.224301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:19.225542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:19.296845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 507/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:19.564838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34293ae-027d-4916-881c-c2f92d1114af None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:19.671271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34293ae-027d-4916-881c-c2f92d1114af None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:19.706137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:19.706778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:19.707060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:19.707342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] PATH_INFO: `/v3/services/image` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:19.707932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:19.709329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:19.718338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:19.719551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:19.720451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 506/1013] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 29 21:22:19 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:19.725319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:19.826277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4228f-feed-4446-808d-da223c73c226 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:19.865734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:19.866352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:19.866629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:19.866898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:19.867520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:19.867820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:19.869201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:19.870349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:19.876744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 508/1014] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:19.882497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:19.939171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:19.991608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762809e8-5d9b-4692-a59b-c3612b051aae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:20.028325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:20.029006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.029286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.029559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.030190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:20.030489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:20.032238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cff8e0-7568-4357-bf00-e7ca34a46fbe None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:20.041914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 507/1015] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:20.043134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:20.083579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711d7c9-a778-49f0-991c-6f6398234d08 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.083766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711d7c9-a778-49f0-991c-6f6398234d08 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.084050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711d7c9-a778-49f0-991c-6f6398234d08 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.085596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 508/1016] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:20.090594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.090879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.091155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.110454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:20.111652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:20.156030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac649d8-5cc2-491d-b6ec-c9c18d8dcbea None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:20.187687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 509/1017] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:20 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:20.198448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:20.199069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.199351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.199624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.200376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] RBAC: Adding query filter params (interface=public, service_id=581b8c7a59824052a32e74f38b1f54df, region_id=RegionOne) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:20.201299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:20.202591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:20.208447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 509/1018] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:22:20 2022] GET /identity/v3/endpoints?service_id=581b8c7a59824052a32e74f38b1f54df&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:20.214575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:20.262510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:20.320437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d8f2-2631-4578-8d7b-fe2730d73533 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:20.357438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:20.357968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.358246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.358517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.359123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:20.360490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:20.367434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 510/1019] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:22:20 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:22:20.457745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:20.544989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:20.598678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66090a1-3880-4f6d-97ba-17ab58bd705a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:20.644883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:20.645466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.645744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.646016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.647218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:20.648694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18698b-e423-450e-beee-0c4022575fc0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:20.650522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 510/1020] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:20.656719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:20.778173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1ac3-18a7-4ce7-90fb-1d80e51c2c0d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:20.791333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:20.829908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:20.830530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:20.830810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:20.831080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:20.831754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:20.833226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b71b5f-432d-46b6-b420-427a2d402672 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:20.876657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 511/1021] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:20 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:22:21.010173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbfb97-4df5-4e55-bf18-ad80e6c26181 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:21.010516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbfb97-4df5-4e55-bf18-ad80e6c26181 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:21.010695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbfb97-4df5-4e55-bf18-ad80e6c26181 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:21.012131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 511/1022] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:21.018254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:21.018751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:21.018875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:21.038059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:21.039277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:21.124899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 512/1023] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:21.392957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbfb97-4df5-4e55-bf18-ad80e6c26181 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:21.499080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbfb97-4df5-4e55-bf18-ad80e6c26181 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:21.530251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:21.537553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:21.538182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:21.538455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:21.538720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:21.539288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:21.540726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:21.549613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:21.550617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:22:21.551471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:21.555623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 512/1024] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:21.556830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:21.647200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:21.660002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abdbdf-3586-4fb1-b441-a0998277292e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:21.693992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:21.694597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:21.694922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:21.695141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:21.695770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:21.696067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:21.697441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:21.702758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 513/1025] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:21.708221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:21.816432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea106c5-6cd7-49ef-b7f0-1fba6f0955c7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:21.850511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:21.851119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:21.851395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:21.851666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:21.852290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:21.852604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:21.853966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa87382-0f56-47ea-94ab-6d9946e2937e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:21.864251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 513/1026] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 29 21:22:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:21.869737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:21.977011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774da873-cf18-4949-aea6-14fe92c09487 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:22.014460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:22.015087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.015365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.015636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.016323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:22.017723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7922a2e-4a58-422f-b010-ae2ea687b608 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:22.038864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 514/1027] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:22.346518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa341f-a532-4ca5-b945-0f817b2c65f2 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.346904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa341f-a532-4ca5-b945-0f817b2c65f2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.347105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa341f-a532-4ca5-b945-0f817b2c65f2 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.348574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 514/1028] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:22.353682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469bdb-ccb5-4ce3-9c4a-77badea549d4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.353969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469bdb-ccb5-4ce3-9c4a-77badea549d4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.354241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469bdb-ccb5-4ce3-9c4a-77badea549d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.373837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c0469bdb-ccb5-4ce3-9c4a-77badea549d4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:22.375026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c0469bdb-ccb5-4ce3-9c4a-77badea549d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:22.379001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:22.455974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 515/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:22.738296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa341f-a532-4ca5-b945-0f817b2c65f2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:22.800895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:22.848983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa341f-a532-4ca5-b945-0f817b2c65f2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:22.865618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497f96-6a4a-44e5-b436-12dd3bee1dee None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.865909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497f96-6a4a-44e5-b436-12dd3bee1dee None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.866196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497f96-6a4a-44e5-b436-12dd3bee1dee None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.868768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 516/1030] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:22.873568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.873851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.874127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.897420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:22.899066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:22.908305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:22.909524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:22.909839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:22.910083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] PATH_INFO: `/v3/services/image` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:22.910672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:22.912882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:22.926094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:22.927208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:22.928070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 515/1031] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 29 21:22:22 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:22.933972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:22.982948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 517/1032] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:23.043559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03bcfa1-84b7-4540-9ce9-8b87699cef54 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:23.067980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:23.083089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:23.084039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:23.084323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:23.084741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:23.085526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] RBAC: Adding query filter params (name=image) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:23.086070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:23.088671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:23.100497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 516/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:23.101952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:23.181250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:23.218369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0111856-a0cd-493c-9680-ebe53b25e512 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:23.254924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:23.255640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:23.255925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:23.256248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:23.256907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] RBAC: Adding query filter params (type=image) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:23.257209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:23.258572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:23.264748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 518/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:23.269885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:23.270029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:23.384547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493f637-7bd1-4303-9747-2830159dd53d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:23.385080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf541-6f5c-4f86-aca9-a77ed1195b21 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:23.420625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:23.421240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:23.421521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:23.421822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:23.422469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:23.423578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:23.424649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93bdfe-1c96-44db-88be-ef7dbabf6196 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:23.431312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:23.432542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:23.433048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:23.433747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:23.435068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:23.436705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:23.439178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 519/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:23.445820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:23.452811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 517/1036] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:23.571099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2e79-96a8-4e37-84ac-fa8639098ac7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:23.618295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:23.618908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:23.619189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:23.619462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:23.620108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:23.621581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99f6f-627d-4cb5-abb1-17f11a467a0a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:23.646534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 520/1037] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:22:23 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:24.070888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96c7c5-b493-4963-8df4-f3c37da9620f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:24.071414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96c7c5-b493-4963-8df4-f3c37da9620f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:24.071414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96c7c5-b493-4963-8df4-f3c37da9620f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:24.072926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 518/1038] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:24.077916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:24.078203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:24.078479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:24.097019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:24.098525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:24.155912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:24.181264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 521/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:24.453728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96c7c5-b493-4963-8df4-f3c37da9620f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:24.501502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:24.564123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96c7c5-b493-4963-8df4-f3c37da9620f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:24.606097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:24.607252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:24.607645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:24.608352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:24.618396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:24.623347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:24.624300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:24.626127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:24.628662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 519/1040] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:22:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:24.630524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:24.738975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78eac-f9b5-4683-ad50-61845774859f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:24.773180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:24.773786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:24.774057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:24.774410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:24.775070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:24.775413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:24.776866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:24.793512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 522/1041] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:22:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:24.793712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:24.904340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6bd82-acbb-4075-a6fe-b48c6f7aae91 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:24.906334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:24.941267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:24.941895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:24.942173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:24.942441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:24.953240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:24.954675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:24.969991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:24.971018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:24.971970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:24.971970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:24.971970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:24.971970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 520/1042] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:24.978461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.028265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.103475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a95d64-2862-40b7-8f27-13fc0ccbc254 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:25.144733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:25.145347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.145622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.145890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.146537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:25.146826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:25.148187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:25.168666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 523/1043] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:25.171077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.296962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb554a62-4807-4b99-9ffb-a90099c785c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:25.335031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:25.335670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.336436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.336436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.345604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:25.347200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-dba80706-6938-48cd-b93f-98831b206ba8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:25.358654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:25.359641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:25.360836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 521/1044] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:22:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:25.367308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.429341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6b465-5f58-4987-a0db-4cb7d02abb30 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.429651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6b465-5f58-4987-a0db-4cb7d02abb30 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.429903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6b465-5f58-4987-a0db-4cb7d02abb30 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.431251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 522/1045] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:25.436440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.436787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.437037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.456056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:25.457399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:25.467810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.479494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da714f4-4643-4d87-bea2-fe8cc09a5041 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:25.481725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.519917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:25.520522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.520992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.521267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.522026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:25.522324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:25.523760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b4600-4095-475b-9c4e-1fe29c49317d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:25.534608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 523/1046] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:25.544970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 524/1047] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:22:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:25.550490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.622571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee5c0-fce4-41ea-a0e6-532c406e25b7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.622920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee5c0-fce4-41ea-a0e6-532c406e25b7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.623120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efee5c0-fce4-41ea-a0e6-532c406e25b7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.624611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 525/1048] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:25.629184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.629466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.629740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.649256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:25.650438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:25.654072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1844ff-a75a-47f2-82b1-17105457ce77 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:25.689486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:25.690094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.690371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.690643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.705863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.726506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 526/1049] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:25.764162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:25.765660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:25.802018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.868176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 524/1050] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:25.874572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:25.914250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.923543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:25.930519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d2b63-af68-426e-a56f-b5906b54d84a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:25.973892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:25.974522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:25.974798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:25.975069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:25.991868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf43f-16fe-413a-9281-bfdb063bf575 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.027165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.027801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.028081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.028351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.039283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.040748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.045790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.047244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:26.056020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:26.057056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:26.058002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:26.058002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:26.058002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:26.058002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 525/1051] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:26.058002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.136608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.147493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 527/1052] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:22:26.148312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.151899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.160931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7105f13-9c00-4fa1-8936-64ec6b50d914 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.196152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.196920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.197073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.197347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.209055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:26.213861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:26.214858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 526/1053] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:26.215733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.251681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b319-6942-4295-b118-4b4342ef0a57 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.287130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.287727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.288006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.288275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.288942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:26.289244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.290617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.301732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 528/1054] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:26.302688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.320101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f26052-a778-4ccc-b28f-f3ce6031344b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.355515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.355731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.356146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.356426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.356810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.369970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.371411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:26.385640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:26.386656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:26.387613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:26.387613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:26.387613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:26.387613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 527/1055] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:26.387613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.404070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.412534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df869409-b876-4978-989d-7d91344fd0b6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.446450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.447057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.447336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.447613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.448283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:26.448612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.449984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.459379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 529/1056] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:26.460166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.497618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b84af7-8013-404b-bbe4-5986d4cf93c3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.531344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.531970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.532246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.532516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.541750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.544611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.559315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baed4-2dce-47a2-923e-67aba56a28f5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:26.561837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:26.562956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:26.563912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 528/1057] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:26.564430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.574196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.597510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.598122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.598443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.598660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.599298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:26.599595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.600985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.613344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 530/1058] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:26.614367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.639726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.682698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359c88-a1b4-4531-bdd8-f27f21c8c513 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.728202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.728888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.729163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.729431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] PATH_INFO: `/v3/users/d791b31c8ed245b5b491af823f85393a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.730584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ecba4-2a69-457b-a977-bec61c5bc27f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.746399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] RBAC: Authorizing `identity:get_user(user_id=d791b31c8ed245b5b491af823f85393a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.748394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.750145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 529/1059] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/users/d791b31c8ed245b5b491af823f85393a => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:26.756187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.770590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.771889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.771889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.771889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.772407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:26.773963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.774192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.788006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 531/1060] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:26.794703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:26.834616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.877836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a779e1-6c38-4a90-ac5e-f0ab4d646c4a None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.914902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:26.926758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.927378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.927661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.927935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.928938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006ce7-2627-41cb-a68e-b4ae739b4e76 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:26.970595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:26.971340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:26.971741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:26.972203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:26.973509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] RBAC: Adding query filter params (role.id=b8f1915970ff43288cf21f586082f269, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:26.974031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:26.976553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:26.984998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 532/1061] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=b8f1915970ff43288cf21f586082f269 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:27.006806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:27.008295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:27.124630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 530/1062] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:27.130508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:27.230976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:27.237426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4576a-50be-4eba-ae57-d737cb4f1e67 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:27.272365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:27.273071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:27.273319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:27.273639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:27.274792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:27.276237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc7169-6aa4-42ff-a3cf-78701d2d37d3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:27.278121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 533/1063] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:27 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:27.285123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:27.313743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:27.393492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b23a1e-441c-4f6f-8f5b-2b81a8763870 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:27.427859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:27.428458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:27.428760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:27.429028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:27.429745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=d5a9d98c5fda4adba100884c852be88d, user.id=d791b31c8ed245b5b491af823f85393a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:27.430046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:27.431457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff133-4faf-499a-8c5b-024dd6ead76e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:27.437495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 531/1064] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:27 2022] GET /identity/v3/role_assignments?user.id=d791b31c8ed245b5b491af823f85393a&scope.project.id=d5a9d98c5fda4adba100884c852be88d&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:28.899841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d9156-c75c-42cc-9cb2-4bcce540f455 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:28.900348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d9156-c75c-42cc-9cb2-4bcce540f455 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:28.900438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d9156-c75c-42cc-9cb2-4bcce540f455 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:28.901761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 534/1065] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:28.907850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914521-0404-4a69-9c6a-5eb3a363460d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:28.908268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914521-0404-4a69-9c6a-5eb3a363460d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:28.908410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914521-0404-4a69-9c6a-5eb3a363460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:28.926882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0f914521-0404-4a69-9c6a-5eb3a363460d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:28.928077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0f914521-0404-4a69-9c6a-5eb3a363460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:28.995311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 532/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:29.270311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d9156-c75c-42cc-9cb2-4bcce540f455 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.303527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c7236-0c74-4346-850a-d3865d989294 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.303815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c7236-0c74-4346-850a-d3865d989294 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.304086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c7236-0c74-4346-850a-d3865d989294 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.305581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 533/1067] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:29.311313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.311590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.311858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.329379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:29.330601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:29.344398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:29.374418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d9156-c75c-42cc-9cb2-4bcce540f455 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.398374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 534/1068] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:29.408233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.408890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.409170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.409439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.420513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:29.425306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:29.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:29.427247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 535/1069] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:29.432218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.529334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59581-ae8b-4e5d-888e-31a48c88fa2a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.563435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.564040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.564317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.564613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.565269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:29.565571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:29.566927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:29.574761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 535/1070] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:29.580400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.605440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:29.666814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.681274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c67871-4ecd-4a51-bb91-6e30f5a4880d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.682312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:29.716892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.717524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.717807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.718096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.728345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:29.729821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:29.744628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:29.745726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:29.746650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:29.746650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:29.746650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:29.746650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 536/1071] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:29.751908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.773173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2a9b8-1a66-447d-9e99-a4503bb0072d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.806517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.807070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.807348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.807615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.818891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:29.823734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:29.824912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:29.826413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 536/1072] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:29.829767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:29.831719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.858684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89192f22-5b92-49e5-bccf-ca28bbf96f73 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.893084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.893693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.893979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.894251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.894895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:29.895198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:29.896567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:29.906222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:29.908939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 537/1073] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:29.914493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:29.938115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec13e5-e830-4b0c-a39a-f95142aad9f4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:29.972320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:29.972948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:29.973223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:29.973494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:29.974147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:29.974452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:29.975806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:29.983744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 537/1074] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:29.989423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.015821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfc7b7-e83f-4a70-9904-60acb9091c65 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.050354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829701de-41e0-41c8-b944-46530d128419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.050978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829701de-41e0-41c8-b944-46530d128419 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.051254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829701de-41e0-41c8-b944-46530d128419 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.051526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829701de-41e0-41c8-b944-46530d128419 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.054205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.059225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829701de-41e0-41c8-b944-46530d128419 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.061493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829701de-41e0-41c8-b944-46530d128419 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-829701de-41e0-41c8-b944-46530d128419 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:30.073235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:30.074422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:30.076763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 538/1075] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:30.080513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829701de-41e0-41c8-b944-46530d128419 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.091789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986a53c-24dc-432d-bd81-4982698da33d None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.126808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.127438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.127712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.127983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] PATH_INFO: `/v3/users/d791b31c8ed245b5b491af823f85393a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.129133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] RBAC: Authorizing `identity:get_user(user_id=d791b31c8ed245b5b491af823f85393a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.130554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.132129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 538/1076] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:29 2022] GET /identity/v3/users/d791b31c8ed245b5b491af823f85393a => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:30.138251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.142002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.186030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829701de-41e0-41c8-b944-46530d128419 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.221144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.221750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.222033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.222299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.223022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:30.223325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.224772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.241093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733280a3-c9d8-437c-9c9d-1d35aad53d59 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.241870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 539/1077] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:30.247939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.274837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27803486-b178-45b4-aa49-3b045496493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.275469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803486-b178-45b4-aa49-3b045496493c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.275751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803486-b178-45b4-aa49-3b045496493c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.276026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27803486-b178-45b4-aa49-3b045496493c None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.277168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27803486-b178-45b4-aa49-3b045496493c None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.278635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27803486-b178-45b4-aa49-3b045496493c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.280279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 539/1078] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:30.286347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27803486-b178-45b4-aa49-3b045496493c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.303358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.348666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316532f7-1e21-4267-ac84-2f338aa6a7aa None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.355085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.385017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.385643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.385922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.386194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.386364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27803486-b178-45b4-aa49-3b045496493c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.397412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.398860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:30.414173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:30.415157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:30.416072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:30.416072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:30.416072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:30.416072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 540/1079] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:30.420965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.421649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.421768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.421932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.422207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d/users/d791b31c8ed245b5b491af823f85393a/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.424073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5a9d98c5fda4adba100884c852be88d, user_id=d791b31c8ed245b5b491af823f85393a, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.425532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.439486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 540/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:22:30 2022] PUT /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d/users/d791b31c8ed245b5b491af823f85393a/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:22:30.532697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6835808-f95a-4185-bac6-b39055e98c60 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.540155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.568994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.569606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.569885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.570158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.570808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:30.571109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.572749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.587113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 541/1081] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:30.593645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.692478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ec37-c56c-4ca0-8f39-6e6024a49c40 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.721064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:30.729297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.729937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.730221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.730498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.738084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.739527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:30.750834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:30.751786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:30.752757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 541/1082] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:30.757910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:30.859297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9a698-bd2a-4cbb-99e5-67b293fcddf8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:30.894696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:30.895311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:30.895593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:30.895868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:30.896579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:30.896907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:30.898361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44b407-faf1-47c0-8c9d-e3e59ce1c18b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:30.910931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 542/1083] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:30.916950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:31.015824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae108481-01a7-4984-853f-c4b96e5a38bc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:31.049981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:31.050636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:31.050915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:31.051189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/b8f1915970ff43288cf21f586082f269` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:31.053322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0446b5664ae0453a8ccd57dfc562eb9b, role_id=b8f1915970ff43288cf21f586082f269)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:31.054741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231606-45ac-4234-bb51-772dbe9f19f4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:31.069183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 542/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:22:30 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0446b5664ae0453a8ccd57dfc562eb9b/roles/b8f1915970ff43288cf21f586082f269 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:22:32.338070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781c0a-ea9b-444f-a04c-750aff282438 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:32.338650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781c0a-ea9b-444f-a04c-750aff282438 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:32.338650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781c0a-ea9b-444f-a04c-750aff282438 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:32.340013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 543/1085] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:32.345565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:32.346008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:32.346110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:32.364774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:32.365958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:32.412877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:32.434096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 543/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:32.716681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e781c0a-ea9b-444f-a04c-750aff282438 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:32.744255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:32.819137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e781c0a-ea9b-444f-a04c-750aff282438 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:32.854746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:32.855410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:32.855758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:32.855962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:32.868463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-77479eab-630e-4cf2-8df5-903ac068f302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:32.873809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:32.874827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:32.875766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 544/1087] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:32.880798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:32.982387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb8d3-9733-4e67-a4a2-6a62a86fc484 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:32.990018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae36b5-3f74-4aa0-823f-b42c02a5d281 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:32.990391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae36b5-3f74-4aa0-823f-b42c02a5d281 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:32.990672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae36b5-3f74-4aa0-823f-b42c02a5d281 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:32.992031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 545/1088] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:32.996979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:32.997277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:32.997552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.015756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:33.016967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:33.018388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.019001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.019277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.019545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.020218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:33.020523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.021898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.030834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 544/1089] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:33.036702 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.080034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.086154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 546/1090] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:22:33.135651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6d5b-ba49-47b0-8214-2163e8e8e8a5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.171056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.171711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.171992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.172265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] PATH_INFO: `/v3/users/d791b31c8ed245b5b491af823f85393a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.173405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] RBAC: Authorizing `identity:get_user(user_id=d791b31c8ed245b5b491af823f85393a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.174833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.176408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 545/1091] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/users/d791b31c8ed245b5b491af823f85393a => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:33.182380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.201340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.284872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd38c9-3d9c-4bda-bfeb-8e6cfb358eff None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.317103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.317701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.317976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.318242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.366120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.386817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.388966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.407204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.472457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07def394-ddf6-4d3e-9562-d8b03397c97c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.485652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.494678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 547/1092] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:33.500654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.509610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.510241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.510517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.510786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.521552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:33.526716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:33.527904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:33.529559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 546/1093] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:33.537206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.619440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761553d-2f5a-4514-8753-65180dbb66db None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.637912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.651478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92763a-b577-405c-b21a-be95b3abca42 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.659353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.660040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.660794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.661073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.662215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.663660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.665374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 548/1094] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:33.672157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.694120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.694799 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.695081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.695429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.696028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:33.696331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.697749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.707164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 547/1095] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:33.712780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.743917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:33.774719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e93f63-fcb0-47d8-b38d-585ee7591aa8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.809263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.809818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.810103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.810377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.811106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=d5a9d98c5fda4adba100884c852be88d, user.id=d791b31c8ed245b5b491af823f85393a) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:33.811413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.812878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.816576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce91b1-ea50-48d3-95d5-18dc4cecb006 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:33.818779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 549/1096] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/role_assignments?user.id=d791b31c8ed245b5b491af823f85393a&scope.project.id=d5a9d98c5fda4adba100884c852be88d&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:22:33.859093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:33.859770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:33.860044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:33.860318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:33.872200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:33.874037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:33.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:33.892249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:33.898682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:33.898682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:33.898682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:33.898682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 548/1097] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:33.899002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:33.962081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:34.033244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae11b9-3599-4510-93c9-68ddf054a3b2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:34.068300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:34.068858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:34.069135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:34.069406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:34.070051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:34.070354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:34.071725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:34.085020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 550/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:34.091536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:34.181355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:34.198350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c622c-98bb-40cd-bb1c-8392e60d56b1 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:34.234684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:34.235324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:34.235612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:34.235886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:34.245061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:34.246516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:22:34.259354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:34.260571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:34.261686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 29 21:22:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:34.269090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:34.437450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadcdf1-5603-4566-b28b-425e84e8bfe7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:34.453218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:34.486778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:34.487390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:34.487666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:34.487935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:34.488697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:34.489000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:34.490475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:34.510630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 551/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 29 21:22:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:34.517748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:34.661454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85fa0-10ab-4aab-a984-92906701dd06 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:34.707154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:34.707673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:34.707952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:34.708222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:34.727156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:34.785113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:34.786559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:34.889405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 550/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:34.895658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:35.005338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e380-542f-44b4-b374-ea45961ca71f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:35.044510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:35.045223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.045502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.045772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.056449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:35.057927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:22:35.078527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:35.079603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:22:35.080551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:22:35.080551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:22:35.080551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:35.080551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:22:35.085533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:35.130664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:35.184034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20492c-03c2-4483-ae89-c528a007bdb8 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:35.218561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:35.219222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.219514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.219854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.220496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:35.220826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:35.222191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:35.233601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:35.239381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:35.260253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:35.341822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62c207-fe03-439d-a192-ae1a9f8903ef None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:35.375466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:35.376087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.376367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.376675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.383589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:35.385100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:22:35.396656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:35.397624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:22:35.398568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:22:35.400745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 553/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:22:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:35.403857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:35.505528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ff6d5-e13f-436b-8370-a11209d1e601 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:35.507921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:35.540971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:35.541624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.541852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.542128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.542822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:35.543129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:35.545112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652e76a-085f-4b0d-ac25-08cacca5c3cb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:35.557163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:22:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:35.563903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:35.646235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:35.669766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee4052-d35e-49b6-a9c3-673d1698a7d2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:35.701919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:35.702514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.702789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.703059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.703778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] RBAC: Adding query filter params (role.id=b8f1915970ff43288cf21f586082f269, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0446b5664ae0453a8ccd57dfc562eb9b) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:35.704085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:35.705535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aeb8-95a4-44c9-9207-760ba53d4e10 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:35.711228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:35 2022] GET /identity/v3/role_assignments?user.id=0446b5664ae0453a8ccd57dfc562eb9b&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=b8f1915970ff43288cf21f586082f269 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:35.875518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8247df-c02c-4f9a-a7be-fc4f1028adec None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.875805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8247df-c02c-4f9a-a7be-fc4f1028adec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.876073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8247df-c02c-4f9a-a7be-fc4f1028adec None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.877525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:35.882934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:35.883214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:35.883486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:35.902070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:35.903752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:35.975376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:36.247732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8247df-c02c-4f9a-a7be-fc4f1028adec None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:36.353691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8247df-c02c-4f9a-a7be-fc4f1028adec None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:36.390199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:36.390833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:36.391106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:36.391376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:36.392483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:36.394088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdb5e4-f5bf-44a5-b979-f7114a6f5ffd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:36.395662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:36.401752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:36.475861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:36.507144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5e5b1-ca56-44af-bb01-6a8a85f1fd93 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:36.542993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:36.543596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:36.544412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:36.544710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:36.545358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:36.546787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6763c5-52fa-4b36-937a-c0703c1215ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:36.568097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:22:36 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:38.515776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ec93-5d22-4a6a-8352-12d4f70da427 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:38.516342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ec93-5d22-4a6a-8352-12d4f70da427 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:38.516342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ec93-5d22-4a6a-8352-12d4f70da427 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:38.517688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:38.522786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:38.523154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:38.523341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:38.541738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:38.542921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:38.609567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:38.887967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82ec93-5d22-4a6a-8352-12d4f70da427 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:38.920378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:38.992928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82ec93-5d22-4a6a-8352-12d4f70da427 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:39.026651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:39.027269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:39.027549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:39.027826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] PATH_INFO: `/v3/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:39.029303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] RBAC: Authorizing `identity:get_role(role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:39.030936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:39.032250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:38 2022] GET /identity/v3/roles/2be7d12a9a644f299290473830d5c537 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:39.038090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:39.096330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:39.136880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67704c2b-9d3b-4115-9a68-33a06d1714c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:39.173076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:39.173698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:39.173972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:39.174263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] PATH_INFO: `/v3/users/90f50b512c2a4da8bd662fce88eb5e3e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:39.187239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] RBAC: Authorizing `identity:get_user(user_id=90f50b512c2a4da8bd662fce88eb5e3e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:39.188693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:39.190311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:39 2022] GET /identity/v3/users/90f50b512c2a4da8bd662fce88eb5e3e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:39.196127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:39.290757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:39.293823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a1ab5-9175-4811-9a84-1b7c6f3d4b26 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:39.328641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:39.329240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:39.329517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:39.329788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:39.396280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:39.397660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:39.492848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:39.496995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:39.502662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:39.600307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca31ac-4f84-4e59-88a3-9cadd0a534dc None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:39.616775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:39.635056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:39.635677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:39.635951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:39.636219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:39.637372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:39.638795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858269a-68b6-4344-aa68-5ca401557672 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:39.640406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:39 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:39.646633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:39.746129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22d13b-75f4-4e93-873c-7e842c658702 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:39.781018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:39.781611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:39.781907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:39.782158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:39.782871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=d5a9d98c5fda4adba100884c852be88d, user.id=90f50b512c2a4da8bd662fce88eb5e3e) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:39.783174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:39.784611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014c99-e653-4e56-bdd7-7d3be8a732f4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:39.789806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:39 2022] GET /identity/v3/role_assignments?user.id=90f50b512c2a4da8bd662fce88eb5e3e&scope.project.id=d5a9d98c5fda4adba100884c852be88d&role.id=2be7d12a9a644f299290473830d5c537 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:41.657881 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92994067-3a26-451b-bf20-2988bdf7d63b None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:41.658415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92994067-3a26-451b-bf20-2988bdf7d63b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:41.658415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92994067-3a26-451b-bf20-2988bdf7d63b None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:41.659774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:41.664920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:41.665326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:41.665458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:41.683750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:41.685084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:41.741305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:41.750131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:42.020627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92994067-3a26-451b-bf20-2988bdf7d63b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:42.120893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92994067-3a26-451b-bf20-2988bdf7d63b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:42.153923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:42.154542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:42.154822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:42.155095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] PATH_INFO: `/v3/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:42.156765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] RBAC: Authorizing `identity:get_role(role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:42.158132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:42.159455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:42 2022] GET /identity/v3/roles/2be7d12a9a644f299290473830d5c537 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:42.165423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:42.263139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6282-e907-48e3-9b63-24fdbc78187c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:42.296637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:42.297261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:42.297542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:42.297817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] PATH_INFO: `/v3/users/90f50b512c2a4da8bd662fce88eb5e3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:42.298924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] RBAC: Authorizing `identity:get_user(user_id=90f50b512c2a4da8bd662fce88eb5e3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:42.300339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:42.301940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:42 2022] GET /identity/v3/users/90f50b512c2a4da8bd662fce88eb5e3e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:42.307649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:42.360877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:42.407638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd45bb-8d23-4f36-9d86-8ef760567cb5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:42.440442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:42.441083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:42.441361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:42.441629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:42.442710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:42.444134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821affc4-11e4-459a-a017-3fae13c1c3bf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:42.445854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:42 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:42.452472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:42.511974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:42.551316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f152d-7c8e-4075-becd-228f4bf39cee None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:42.584122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:42.584796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:42.585076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:42.585349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d/users/90f50b512c2a4da8bd662fce88eb5e3e/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:42.587466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5a9d98c5fda4adba100884c852be88d, user_id=90f50b512c2a4da8bd662fce88eb5e3e, role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:42.588912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c377f-776f-4bb0-a851-12a620f9fb04 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:42.601435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:22:42 2022] PUT /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d/users/90f50b512c2a4da8bd662fce88eb5e3e/roles/2be7d12a9a644f299290473830d5c537 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:22:44.499250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd03f-5e30-40a9-ac46-f9145b3f16d0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:44.499813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd03f-5e30-40a9-ac46-f9145b3f16d0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:44.499813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccd03f-5e30-40a9-ac46-f9145b3f16d0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:44.501186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:44.506414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:44.506849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:44.506949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:44.526080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:44.527255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:44.592027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:44.871403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd03f-5e30-40a9-ac46-f9145b3f16d0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:44.971594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccd03f-5e30-40a9-ac46-f9145b3f16d0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:44.991511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:45.005365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:45.005982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:45.006259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:45.006531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] PATH_INFO: `/v3/roles/2be7d12a9a644f299290473830d5c537` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:45.007903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] RBAC: Authorizing `identity:get_role(role_id=2be7d12a9a644f299290473830d5c537)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:45.009293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:45.010617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:44 2022] GET /identity/v3/roles/2be7d12a9a644f299290473830d5c537 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:45.017143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:45.068461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:45.117686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225001-07d9-4691-b6c7-9eb827789425 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:45.151505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:45.152136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:45.152415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:45.152774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] PATH_INFO: `/v3/users/90f50b512c2a4da8bd662fce88eb5e3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:45.154495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] RBAC: Authorizing `identity:get_user(user_id=90f50b512c2a4da8bd662fce88eb5e3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:45.156210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:45.157889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:45 2022] GET /identity/v3/users/90f50b512c2a4da8bd662fce88eb5e3e => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:45.163755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:45.263285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c649fe-f221-45b3-9d1c-53f2639ff47d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:45.297073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:45.297669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:45.297943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:45.298212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:45.357648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:45.370434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:45.372357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:45.470101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:45.476189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:45.574985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937c252-a379-4cc7-8f57-c9f3480bc1e3 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:45.597761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:45.612891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:45.613512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:45.613788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:45.614059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] PATH_INFO: `/v3/projects/d5a9d98c5fda4adba100884c852be88d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:45.615169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a9d98c5fda4adba100884c852be88d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:45.616603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d862e-c8c8-44bb-bc44-fbaa399048f3 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:45.618226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:45 2022] GET /identity/v3/projects/d5a9d98c5fda4adba100884c852be88d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:45.626916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:45.721420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:45.739197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7846-96ac-4c63-87c2-820236345df2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:45.782260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:45.782864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:45.783148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:45.783455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:45.784185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] RBAC: Adding query filter params (role.id=2be7d12a9a644f299290473830d5c537, scope.project.id=d5a9d98c5fda4adba100884c852be88d, user.id=90f50b512c2a4da8bd662fce88eb5e3e) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:45.784491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:45.785971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92402eda-158a-4ac9-b939-95954c0f6e6c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:45.792296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:45 2022] GET /identity/v3/role_assignments?user.id=90f50b512c2a4da8bd662fce88eb5e3e&scope.project.id=d5a9d98c5fda4adba100884c852be88d&role.id=2be7d12a9a644f299290473830d5c537 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:47.749766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063b1-d7b4-4b1b-bf8a-1e12662568f9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:47.750560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063b1-d7b4-4b1b-bf8a-1e12662568f9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:47.750560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217063b1-d7b4-4b1b-bf8a-1e12662568f9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:47.751664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:47.757868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162607-db01-4a28-8424-545312f87b4f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:47.758365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162607-db01-4a28-8424-545312f87b4f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:47.758655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162607-db01-4a28-8424-545312f87b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:47.778532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4e162607-db01-4a28-8424-545312f87b4f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:47.779706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4e162607-db01-4a28-8424-545312f87b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:47.846697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:48.123069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217063b1-d7b4-4b1b-bf8a-1e12662568f9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:48.224546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217063b1-d7b4-4b1b-bf8a-1e12662568f9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:48.258805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:48.259435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:48.259714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:48.259985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:48.261120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:48.262550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5af2dc-31ff-4e44-8bca-34f01c5fd122 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:48.264327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:48.270165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e162607-db01-4a28-8424-545312f87b4f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:48.358299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:48.377980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e162607-db01-4a28-8424-545312f87b4f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:48.415054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:48.415655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:48.415935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:48.416209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:48.416896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:48.418352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58cd89-d2d0-4c9e-93f7-c4139c9f3cc3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:48.445908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:22:48 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:50.388663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c7e8-efb6-4882-aee6-ee0869ab80f0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:50.389223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c7e8-efb6-4882-aee6-ee0869ab80f0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:50.389223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c7e8-efb6-4882-aee6-ee0869ab80f0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:50.390568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:50.395588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:50.395986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:50.396134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:50.414781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:50.415951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:50.485377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:50.756484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c7e8-efb6-4882-aee6-ee0869ab80f0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:50.815973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:50.866167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c7e8-efb6-4882-aee6-ee0869ab80f0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:50.900185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:50.900856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:50.901138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:50.901408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:50.903058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:50.904508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f24ad-0fcd-41db-b79b-927ab24035e1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:50.906391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:22:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:50.912153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:50.973838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:51.015297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be8d95-c157-41a5-9af9-a802eec191da None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:51.053064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:51.053670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:51.053946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:51.054216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:51.054865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:51.056283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef12b8-6550-4da2-ae93-87e5517c50cd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:51.078422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:22:50 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:53.048119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c765d96-e377-461e-b1b2-e834fc7fd754 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.048673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c765d96-e377-461e-b1b2-e834fc7fd754 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.048673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c765d96-e377-461e-b1b2-e834fc7fd754 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.050035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:53.055077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.055486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.055603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.073505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:53.074678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:53.141310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:53.415761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c765d96-e377-461e-b1b2-e834fc7fd754 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:53.516249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c765d96-e377-461e-b1b2-e834fc7fd754 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:53.539973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:53.550770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:53.551399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.551679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.551951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.562353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:53.567134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:53.568235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:53.569264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:53.574809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:53.624401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:53.676086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee5845-d729-45f4-8e15-3aa22af1d20b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:53.711378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:53.711979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.712257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.712529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.713215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:53.713515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:53.714900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:53.722825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:53.728802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:53.837521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb19ddd-fafc-4988-a549-5d267e05ede4 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:53.872503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:53.873165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.873437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.873704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] PATH_INFO: `/v3/users/077f3be58c4c4c78af0581e596920686` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.888035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] RBAC: Authorizing `identity:get_user(user_id=077f3be58c4c4c78af0581e596920686)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:53.889484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:53.891411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:53 2022] GET /identity/v3/users/077f3be58c4c4c78af0581e596920686 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:53.897344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:53.998700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28df54-45d1-41c5-91ef-a415f9cb0a7d None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:54.011203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.034348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:54.034954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.035237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.035510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.102133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:54.103596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:54.201694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:22:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:22:54.207906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:54.250837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.307605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be75abd-7a31-4284-8c10-0ef1f964d556 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:54.341679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:54.342295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.342572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.342840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] PATH_INFO: `/v3/projects/80b3515a0bda4fa7ab72c688017a868d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.343953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] RBAC: Authorizing `identity:get_project(project_id=80b3515a0bda4fa7ab72c688017a868d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:54.345403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916efb2c-29ed-458e-9da6-ae834173943a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:54.347026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:54 2022] GET /identity/v3/projects/80b3515a0bda4fa7ab72c688017a868d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:54.353335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:54.367471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.451815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040bdde-ce97-424d-ab23-d3bdc9ef2fc7 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:54.485546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:54.486144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.486422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.486694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.487420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=80b3515a0bda4fa7ab72c688017a868d, user.id=077f3be58c4c4c78af0581e596920686) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:54.487723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:54.489181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b3ca9-125a-4879-96b1-1c71c3cd65cf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:54.494443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:22:54 2022] GET /identity/v3/role_assignments?user.id=077f3be58c4c4c78af0581e596920686&scope.project.id=80b3515a0bda4fa7ab72c688017a868d&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:56.412862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf595-1a91-41eb-883c-abc483ef6b99 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:56.413401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf595-1a91-41eb-883c-abc483ef6b99 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:56.413792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3bf595-1a91-41eb-883c-abc483ef6b99 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:56.416195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:56.422000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:56.422457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:56.422553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:56.441440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:56.442629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:56.510134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:56.782821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bf595-1a91-41eb-883c-abc483ef6b99 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:56.888468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3bf595-1a91-41eb-883c-abc483ef6b99 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:56.900125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:56.923891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:56.924676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:56.924951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:56.925219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:56.936065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:22:56.941618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:22:56.942786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:22:56.943678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:22:56.948940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:56.957731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:57.049139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27c908-2db2-4dcf-ba0c-d627c1a5dbdc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:57.082921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:57.083533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:57.083810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:57.084082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:57.084775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:57.085078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:57.086439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:57.094808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:57.100874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:57.199977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479415-ec44-43e8-ac25-7d6c773af21f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:57.232974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:57.233589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:57.233864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:57.234131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] PATH_INFO: `/v3/users/077f3be58c4c4c78af0581e596920686` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:57.235317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] RBAC: Authorizing `identity:get_user(user_id=077f3be58c4c4c78af0581e596920686)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:57.236760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:57.238339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:22:57 2022] GET /identity/v3/users/077f3be58c4c4c78af0581e596920686 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:57.244393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:57.310796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:57.344178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f1de3-569a-4078-a37f-808a1116ba52 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:57.378299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:57.378927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:57.379202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:57.379476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] PATH_INFO: `/v3/projects/80b3515a0bda4fa7ab72c688017a868d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:57.380597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] RBAC: Authorizing `identity:get_project(project_id=80b3515a0bda4fa7ab72c688017a868d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:57.382041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b2f46-adbf-40e4-8eef-724aba3eee7c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:57.383688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:22:57 2022] GET /identity/v3/projects/80b3515a0bda4fa7ab72c688017a868d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:57.389696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:57.431314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:57.490509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81219061-cda1-4cc8-8290-0e9467e7950e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:57.524197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:57.524921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:57.525203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:57.525499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] PATH_INFO: `/v3/projects/80b3515a0bda4fa7ab72c688017a868d/users/077f3be58c4c4c78af0581e596920686/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:57.528016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] RBAC: Authorizing `identity:create_grant(project_id=80b3515a0bda4fa7ab72c688017a868d, user_id=077f3be58c4c4c78af0581e596920686, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:57.529602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e52b2-92e6-470e-8812-fa931b43a7cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:57.543058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:22:57 2022] PUT /identity/v3/projects/80b3515a0bda4fa7ab72c688017a868d/users/077f3be58c4c4c78af0581e596920686/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:22:59.509584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffca0f-c13f-4aa7-9cad-69d756e40b46 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:59.510138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffca0f-c13f-4aa7-9cad-69d756e40b46 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:59.510138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffca0f-c13f-4aa7-9cad-69d756e40b46 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:59.511544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:59.516631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:59.517011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:59.517174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:59.535201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:59.536387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:59.601417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:22:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:22:59.873710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffca0f-c13f-4aa7-9cad-69d756e40b46 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:59.976111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffca0f-c13f-4aa7-9cad-69d756e40b46 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.009354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.009971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.010248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.010518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.013692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:00.022995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:23:00.028360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:23:00.031152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:23:00.033225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:23:00.035954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:00.062839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:00.137398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ad798-b8d4-443c-8fee-5819cb9272bb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.172767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.173381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.173661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.173937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.174602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:00.174908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:00.176278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:00.184400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:00.190383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:00.289725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aa666-d70c-4627-88dc-3351d7526ef2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.326986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.327627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.327904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.328177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] PATH_INFO: `/v3/users/077f3be58c4c4c78af0581e596920686` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.329315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] RBAC: Authorizing `identity:get_user(user_id=077f3be58c4c4c78af0581e596920686)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:00.330753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:00.332344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:23:00 2022] GET /identity/v3/users/077f3be58c4c4c78af0581e596920686 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:00.338389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:00.430931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:00.441181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07b26d-68e9-48f5-9805-3e0ae16b1984 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.474381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.474967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.475246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.475518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.541128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:00.542567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:00.630366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:00.642023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:23:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:23:00.648002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:00.710125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:00.750228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fec0e-92fe-422d-bec6-4cdccce44e46 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.783747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.784379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.784694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.784972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] PATH_INFO: `/v3/projects/80b3515a0bda4fa7ab72c688017a868d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.786088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] RBAC: Authorizing `identity:get_project(project_id=80b3515a0bda4fa7ab72c688017a868d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:00.787528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924b619-45b0-40c4-bd39-2980a0bb9beb None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:00.789204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:23:00 2022] GET /identity/v3/projects/80b3515a0bda4fa7ab72c688017a868d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:00.795889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:00.895766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c7a00-30f9-456a-8e18-4df7d52a0e83 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:00.932756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:00.933357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:00.933701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:00.934473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:00.935295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=80b3515a0bda4fa7ab72c688017a868d, user.id=077f3be58c4c4c78af0581e596920686) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:00.935592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:00.937049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f259ba-4413-48de-8246-848736e4f6b1 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:00.942714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:23:00 2022] GET /identity/v3/role_assignments?user.id=077f3be58c4c4c78af0581e596920686&scope.project.id=80b3515a0bda4fa7ab72c688017a868d&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:02.942961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102a4-877b-446e-8794-bdfb095c9278 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:02.943538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102a4-877b-446e-8794-bdfb095c9278 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:02.943538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1102a4-877b-446e-8794-bdfb095c9278 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:02.944953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:02.950633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:02.951077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:02.951176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:02.969881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:02.971056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:03.031911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:03.049518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:03.320829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1102a4-877b-446e-8794-bdfb095c9278 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:03.422002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1102a4-877b-446e-8794-bdfb095c9278 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:03.451954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:03.457321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:03.457926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:03.458203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:03.458474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:03.524280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:03.525750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:03.622797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:23:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:23:03.628764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:03.734764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68395b-8a93-4b27-b045-944563e1b2a8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:03.772784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:03.773414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:03.773777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:03.773989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:03.781753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:03.783262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-31fdda09-fe69-40b9-9d9a-fb700ed20001 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:23:03.796723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:23:03.797866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:23:03.799185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:23:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:23:03.804252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:03.837746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:03.919118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50793-b40c-4bbc-9102-a158c67563d4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:03.954403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:03.955099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:03.955277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:03.955546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:03.956189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:03.956491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:03.957897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31330ef2-73a4-43f4-b3d6-c6dc74480ede None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:03.965470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 29 21:23:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:05.856668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3056-17be-491b-b54e-2ec154bf7cc9 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:05.857228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3056-17be-491b-b54e-2ec154bf7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:05.857228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3056-17be-491b-b54e-2ec154bf7cc9 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:05.858584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:05.863991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb96a0-967a-4a9d-8b42-906c2fbdd7b8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:05.864396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb96a0-967a-4a9d-8b42-906c2fbdd7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:05.864543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb96a0-967a-4a9d-8b42-906c2fbdd7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:05.883205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f3eb96a0-967a-4a9d-8b42-906c2fbdd7b8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:05.884379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f3eb96a0-967a-4a9d-8b42-906c2fbdd7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:05.952359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:06.216333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae3056-17be-491b-b54e-2ec154bf7cc9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:06.262644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:06.317674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae3056-17be-491b-b54e-2ec154bf7cc9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:06.352243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:06.352873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:06.353147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:06.353412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:06.354071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:06.355456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4e8ba-2b2b-4cc3-b123-6cf433dc186f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:06.371160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Aug 29 21:23:06 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:23:08.307880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f41d7-714d-4af0-8338-7a499de6dd72 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:08.309295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f41d7-714d-4af0-8338-7a499de6dd72 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:08.309295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f41d7-714d-4af0-8338-7a499de6dd72 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:08.310404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:08.315738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:08.316160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:08.316276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:08.334378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:08.335577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:08.402515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:08.672169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f41d7-714d-4af0-8338-7a499de6dd72 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:08.699209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:08.775273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f41d7-714d-4af0-8338-7a499de6dd72 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:08.810346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:08.810968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:08.811245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:08.811519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] PATH_INFO: `/v3/domains/83700568a3db4744b1e56c7e1ce92e4f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:08.821941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] RBAC: Authorizing `identity:get_domain(domain_id=83700568a3db4744b1e56c7e1ce92e4f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:08.823423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660c3e4-2252-461e-9261-2ac92da1effa None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:08.825601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 29 21:23:08 2022] GET /identity/v3/domains/83700568a3db4744b1e56c7e1ce92e4f => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:08.831502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:08.865277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:08.935643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bc80a-0f34-4f9d-98c4-0598fd8fd146 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:08.969843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:08.970453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:08.970733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:08.971004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:08.971650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:08.973114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dffa55-de80-4460-8f06-6b991b036afe None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:09.000576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:23:08 2022] POST /identity/v3/projects => generated 361 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:23:10.937424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19fd07-7a0c-48da-85eb-8fc1b04050dd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:10.937982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19fd07-7a0c-48da-85eb-8fc1b04050dd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:10.937982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19fd07-7a0c-48da-85eb-8fc1b04050dd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:10.939392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:10.944772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:10.945297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:10.963109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:10.964920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:11.030343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:23:11.312437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19fd07-7a0c-48da-85eb-8fc1b04050dd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:11.398982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:11.416753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19fd07-7a0c-48da-85eb-8fc1b04050dd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:11.450297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:11.450920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:11.451202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:11.451477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] PATH_INFO: `/v3/domains/83700568a3db4744b1e56c7e1ce92e4f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:11.452621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=83700568a3db4744b1e56c7e1ce92e4f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:11.454056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79094a-0e28-4e75-a0a5-cfb5c78b4591 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:11.455651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 29 21:23:11 2022] GET /identity/v3/domains/83700568a3db4744b1e56c7e1ce92e4f => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:11.461531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:11.479193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:11.563131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9680a7c-726b-4b24-a669-45d344579596 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:11.598403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:11.599002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:11.599279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:11.599549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:11.600207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:11.601670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4e9cc-5582-411f-86f1-f099d135c789 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:11.624727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:23:11 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:23:13.574150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad57fb-6986-42e6-80ef-1151c752689d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:13.574695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad57fb-6986-42e6-80ef-1151c752689d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:13.574695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad57fb-6986-42e6-80ef-1151c752689d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:13.576038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:13.581307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:13.581766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:13.581858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:13.600497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:13.601729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:13.668464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:13.941466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad57fb-6986-42e6-80ef-1151c752689d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:14.059771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad57fb-6986-42e6-80ef-1151c752689d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:14.096529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:14.097195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:14.097476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:14.097748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:14.109292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:23:14.114174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:23:14.115327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:23:14.116195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:23:14.121620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:14.123979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:14.226991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f064c-d164-41f8-bae6-b41d51d479d3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:14.260034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:14.260664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:14.261444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:14.261733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:14.262399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:14.262698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:14.264072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:14.272513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:14.278894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:14.295328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:14.378842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94556a2c-ee9d-4834-ad0b-bb5cf134d295 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:14.418980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:14.419626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:14.419900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:14.420179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] PATH_INFO: `/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:14.433700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] RBAC: Authorizing `identity:get_user(user_id=f46e54d78bbd4f09a464bbdd3a6a7c3a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:14.435176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:14.436888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:23:14 2022] GET /identity/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:14.442934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:14.504236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:14.543731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d94254-547e-402d-bc97-aa9c79d654d2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:14.577439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:14.578053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:14.578337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:14.578610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:14.654115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:14.655738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:14.719787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:14.760605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:23:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:23:14.767015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:14.840160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:14.871782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9193ab-ab9f-4776-8790-ee8fef5719f8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:14.908088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:14.908681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:14.908965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:14.909239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] PATH_INFO: `/v3/projects/0aeede03b6134d63a784feff488e486e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:14.910351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=0aeede03b6134d63a784feff488e486e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:14.911779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e5310-b530-45f2-93ae-1c339eae6ca7 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:14.913443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:23:14 2022] GET /identity/v3/projects/0aeede03b6134d63a784feff488e486e => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:23:14.920177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:15.019785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a891a5-c3c7-453e-8061-695215dccbd7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:15.052764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:15.053358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:15.053638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:15.053907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:15.054627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=0aeede03b6134d63a784feff488e486e, user.id=f46e54d78bbd4f09a464bbdd3a6a7c3a) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:15.054934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:15.056344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddc595-f697-4e77-a6a8-a3fc0d0cd96f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:15.062524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:23:14 2022] GET /identity/v3/role_assignments?user.id=f46e54d78bbd4f09a464bbdd3a6a7c3a&scope.project.id=0aeede03b6134d63a784feff488e486e&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:16.970791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546045-ba1e-4415-8134-f9a3b24280b8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:16.971339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546045-ba1e-4415-8134-f9a3b24280b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:16.971339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7546045-ba1e-4415-8134-f9a3b24280b8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:16.972689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:16.979463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:16.979930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:16.980048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:16.999012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:17.000195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:17.023150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:17.068776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:17.338976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7546045-ba1e-4415-8134-f9a3b24280b8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:17.438667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7546045-ba1e-4415-8134-f9a3b24280b8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:17.472827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:17.473439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:17.473714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:17.473982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:17.481615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:17.484806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:23:17.489704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:23:17.490921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:23:17.491974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:23:17.497059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:17.592373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6146f4-0fca-4963-8d13-6c4ada51a9b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:17.625016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:17.625617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:17.625895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:17.626167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:17.626819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:17.627117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:17.628469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:17.637081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:17.642872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:17.740696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc22456-450a-4fa1-aa41-dda9d5b63bc2 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:17.772818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:17.773422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:17.773695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:17.773961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] PATH_INFO: `/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:17.775050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] RBAC: Authorizing `identity:get_user(user_id=f46e54d78bbd4f09a464bbdd3a6a7c3a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:17.777027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:17.778576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:23:17 2022] GET /identity/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:17.784736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:17.807004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:17.882967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666353-f92b-437b-a548-db6a221bb3cf None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:17.916009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:17.916653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:17.916935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:17.917868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] PATH_INFO: `/v3/projects/0aeede03b6134d63a784feff488e486e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:17.918997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] RBAC: Authorizing `identity:get_project(project_id=0aeede03b6134d63a784feff488e486e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:17.920410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e596-5ecc-4101-bd0b-984702bb48b9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:17.922071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:23:17 2022] GET /identity/v3/projects/0aeede03b6134d63a784feff488e486e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:17.927885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:17.992015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:18.024412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a360f-35b1-4c04-b403-0b78ceb42d92 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:18.058435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:18.059087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:18.059366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:18.059635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] PATH_INFO: `/v3/projects/0aeede03b6134d63a784feff488e486e/users/f46e54d78bbd4f09a464bbdd3a6a7c3a/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:18.061631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] RBAC: Authorizing `identity:create_grant(project_id=0aeede03b6134d63a784feff488e486e, user_id=f46e54d78bbd4f09a464bbdd3a6a7c3a, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:18.063070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee36aa-aba5-4a8f-af84-313862fb2113 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:18.075807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:23:17 2022] PUT /identity/v3/projects/0aeede03b6134d63a784feff488e486e/users/f46e54d78bbd4f09a464bbdd3a6a7c3a/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:23:19.963534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207be-04fc-4a3d-a03d-18b0fcc47803 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:19.964750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207be-04fc-4a3d-a03d-18b0fcc47803 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:19.964750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec207be-04fc-4a3d-a03d-18b0fcc47803 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:19.966007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:23:19.971359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:19.971809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:19.971906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:19.990157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:23:19.991320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:23:20.056253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:23:20.320814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec207be-04fc-4a3d-a03d-18b0fcc47803 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:20.408392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:20.421983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec207be-04fc-4a3d-a03d-18b0fcc47803 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:20.455033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:20.455655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:20.455936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:20.456266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:20.466988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:23:20.471788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:23:20.472930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:23:20.474048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:23:20.478939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:20.534435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:20.574649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b52f-bc8a-4535-b22a-3b8ce55b92d6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:20.607801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:20.608387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:20.608686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:20.608957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:20.609607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:20.609907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:20.611252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:20.618951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:20.625357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:20.723455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6aaff-1d09-4aac-b48c-018e5cb5e8e7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:20.764572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:20.765211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:20.765489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:20.765760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] PATH_INFO: `/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:20.766858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] RBAC: Authorizing `identity:get_user(user_id=f46e54d78bbd4f09a464bbdd3a6a7c3a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:20.768274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:20.769860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 29 21:23:20 2022] GET /identity/v3/users/f46e54d78bbd4f09a464bbdd3a6a7c3a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:20.775685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:20.873076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654f2d-a9d4-4264-a162-94fb5da14db3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:20.892889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:20.908135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:20.908781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:20.909065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:20.909338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:20.976279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:20.977759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:21.076610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:23:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:23:21.082660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:21.100925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:21.181896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462e0d-ed74-4d8e-84e1-6a4f712870e6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:21.215134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:21.215753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:21.216113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:21.216386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] PATH_INFO: `/v3/projects/0aeede03b6134d63a784feff488e486e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:21.217527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] RBAC: Authorizing `identity:get_project(project_id=0aeede03b6134d63a784feff488e486e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:21.218943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eed64-b57e-4b55-be04-e96c62775357 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:21.220565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 29 21:23:21 2022] GET /identity/v3/projects/0aeede03b6134d63a784feff488e486e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:21.226713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:23:21.248459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:21.326763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41620b-fc8d-4961-9d7d-e0706b83b99e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:23:21.359753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:23:21.360342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:23:21.360645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:23:21.360926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:23:21.361646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=0aeede03b6134d63a784feff488e486e, user.id=f46e54d78bbd4f09a464bbdd3a6a7c3a) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:23:21.361957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:23:21.363388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad0ab4-4f81-4953-8b69-5187c0d48943 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:23:21.369112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:23:21 2022] GET /identity/v3/role_assignments?user.id=f46e54d78bbd4f09a464bbdd3a6a7c3a&scope.project.id=0aeede03b6134d63a784feff488e486e&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:23:31.211524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=58 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:31.366465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=58 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:17.730028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fe79-9098-445b-b927-3462eb1cd5d7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:17.731133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fe79-9098-445b-b927-3462eb1cd5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:17.731133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450fe79-9098-445b-b927-3462eb1cd5d7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:17.732747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:24:17.737967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:17.738458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:17.738574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:17.759561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:17.761128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:17.861755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:18.140205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450fe79-9098-445b-b927-3462eb1cd5d7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:18.311694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450fe79-9098-445b-b927-3462eb1cd5d7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:18.379067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:18.379828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:18.380198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:18.380540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:18.404331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:18.406508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:18.426642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:18.427917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:18.429103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:18.429103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:18.429103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:18.430486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:18.440734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:18.574313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bac66c-4174-4877-8e66-f862b9f9fb77 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:18.619021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:18.629623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:18.630624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:18.631114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:18.631598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:18.632481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:18.633083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:18.634989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e8b22-3d77-42b2-a2d6-2d67b4b030ad None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:18.657242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:18.662574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:18.739274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:18.824206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135b6a-13ca-4eb1-87e0-f88c308545b0 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:18.869804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:18.870055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:18.870280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:18.870528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:18.871283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:18.872990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5fa5f-0f49-4708-b7dd-5de576f8a6a0 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:18.908863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 29 21:24:18 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:24:21.078966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a71c-9497-4ef0-b689-7ff3120da056 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:21.079821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a71c-9497-4ef0-b689-7ff3120da056 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:21.079821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a71c-9497-4ef0-b689-7ff3120da056 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:21.082027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:21.088125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:21.088865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:21.088865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:21.110258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:21.111706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:21.202710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:21.503521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a71c-9497-4ef0-b689-7ff3120da056 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:21.637816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a71c-9497-4ef0-b689-7ff3120da056 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:21.683765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:21.684890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:21.685324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:21.685802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:21.701136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:21.707009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:21.709298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:21.710849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:21.717582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:21.768263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:21.853056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b8588-f780-4b36-adc9-097a3fe4c74a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:21.913581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:21.914359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:21.914647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:21.915667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:21.916411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:21.917028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:21.918626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:21.931786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:21.941596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:22.000414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:22.067304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c01da-4101-44a9-ae3b-fe4cf2c8164c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:22.110872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:22.112174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:22.112467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:22.112821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:22.126704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:22.128260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:22.150021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:22.151242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:22.152190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:22.152190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:22.152190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:22.152190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:22.156412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:22.294037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:22.297492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accfc1f-349a-460c-84da-6c6d93d29c71 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:22.353243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:22.353927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:22.354249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:22.354461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:22.355108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:22.355408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:22.357063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:22.375201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:22.382373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:22.509766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ef52c-c8e0-4826-ac95-21ed236c5fe5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:22.511475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:22.557565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:22.558371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:22.558974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:22.559316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:22.569315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:22.571141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:22.585227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:22.586514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:22.588259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:22.591170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:22.592768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:22.722198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcef35-b145-478d-8567-4093c6427a02 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:22.769125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:22.769792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:22.770168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:22.770444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:22.771291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:22.771809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:22.773556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:22.795544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:22.803163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:22.963062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d51ad-ebc8-45f2-bbab-eac150224c9e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:23.028555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:23.029222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:23.029529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:23.029784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:23.082098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:23.138171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:23.139614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:23.251714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:24:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:24:23.257702 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:23.284637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:23.384909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ccd96-311e-4839-a0b3-0b25bd5ae2f6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:23.426524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:23.427338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:23.427778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:23.428151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:23.441450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:23.443142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:23.461483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:23.464210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:23.472377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:23.472377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:23.472377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:23.472377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:23.474133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:23.555202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:23.606765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf280c-d6e6-441b-b2ee-77b6e96a2886 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:23.650954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff839347-8276-421d-808f-6698d20174ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:23.651568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839347-8276-421d-808f-6698d20174ae None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:23.651850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839347-8276-421d-808f-6698d20174ae None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:23.652116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff839347-8276-421d-808f-6698d20174ae None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:23.653057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff839347-8276-421d-808f-6698d20174ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:23.653389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff839347-8276-421d-808f-6698d20174ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:23.654944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff839347-8276-421d-808f-6698d20174ae None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:23.673923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:23.675305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:23.779576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:23.817843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597d16e-905d-4e64-90ed-cb1fc007e8c5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:23.861456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:23.862435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:23.862885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:23.863206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:23.874623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:23.876955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:23.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:23.900868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:23.902034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:23.906458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff839347-8276-421d-808f-6698d20174ae None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:24.051108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff839347-8276-421d-808f-6698d20174ae None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:24.089155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:24.105411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:24.106119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:24.106503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:24.106745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:24.107470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:24.107757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:24.109257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d509c1-b9d0-4f9d-b98f-0323dcf14f11 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:24.129786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:24.130010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:24.290327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961e44e-b023-46c8-b466-bef806ee4dde None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:24.335885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:24.349125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:24.350264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:24.350592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:24.353771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:24.353771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0697bbbe44c74c538761737601ceeb10) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:24.353771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:24.355363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16403e-92b0-4e73-aa81-715ce269da6c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:24.362288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:24:24 2022] GET /identity/v3/role_assignments?user.id=0697bbbe44c74c538761737601ceeb10&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:26.369600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a50d6-e96c-4de5-afcf-c00e8e6b4f78 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:26.370167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a50d6-e96c-4de5-afcf-c00e8e6b4f78 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:26.370167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a50d6-e96c-4de5-afcf-c00e8e6b4f78 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:26.371570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:26.378965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:26.378965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:26.378965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:26.408440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:26.409669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:26.504617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:26.782606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a50d6-e96c-4de5-afcf-c00e8e6b4f78 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:26.921528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a50d6-e96c-4de5-afcf-c00e8e6b4f78 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:26.969661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:26.970297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:26.970580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:26.970855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:26.983304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:26.989091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:26.990419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:26.991397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:26.998282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:27.117289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c6422-ddcb-4627-b6c9-da1de0de7762 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:27.146551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:27.157917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:27.158533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:27.158813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:27.159085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:27.159744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:27.160048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:27.161438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:27.175993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:27.177455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:27.228194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:27.299245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff777b0d-0176-4747-9b8e-4379107f8c6e None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:27.358891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:27.360145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:27.360701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:27.361175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:27.382644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:27.385585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:27.420273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:27.421440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:27.422396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:27.422396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:27.422396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:27.422396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:27.429350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:27.546632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd292c20-abb4-4ea8-99c6-c02b14d177b6 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:27.584214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:27.584977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:27.585284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:27.585621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:27.586331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:27.587130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:27.588519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:27.603221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:27.609125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:27.715950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:27.724208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bccc-772a-4f4d-bf54-d3f0994a6260 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:27.761546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:27.762209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:27.762445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:27.762722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:27.775216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:27.776793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:27.790358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:27.791689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:27.793003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:27.798692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:27.853479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:27.921937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4ca63-b649-4d4b-9427-0adaf40cfa3f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:27.959929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:27.960539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:27.960849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:27.961120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:27.962648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:27.962951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:27.964380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:27.989423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:27.990364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:28.128432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca227-ea5b-4536-9439-d784456de6f4 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:28.180751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:28.181118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:28.181297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:28.181716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:28.199217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:28.201812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:28.221216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:28.222436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:28.223878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:28.223878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:28.223878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:28.223878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:28.228427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:28.357449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb8a51-d02b-4f20-8135-0c7eb29ada77 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:28.361409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:28.408858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:28.409776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:28.410214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:28.410671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:28.411782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:28.412318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:28.415531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:28.434131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:28.442502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:28.456429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:28.618713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc68a96-cf3e-464a-9dc7-95840e8ffcd5 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:28.674565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:28.675204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:28.675476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:28.675749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:28.683392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:28.684859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:28.697346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:28.698935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:28.700270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:28.705378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:28.833317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b95f8-954e-42bf-a4f2-b135908e8942 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:28.877691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:28.878215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:28.878490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:28.878758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:28.879448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:28.879755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:28.881225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98454a4-413f-4389-b39c-f6b79014d108 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:28.899641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:28.901405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:28.982244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:29.034182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c1745-9a8a-4d73-8022-73d6f2cfdfba None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:29.079518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:29.080273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:29.080565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:29.080880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0697bbbe44c74c538761737601ceeb10/roles/f239192299174e29a4672c9df068e024` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:29.100473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0697bbbe44c74c538761737601ceeb10, role_id=f239192299174e29a4672c9df068e024)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:29.102760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3642-cb63-4e16-8ea2-e0159fbf56db None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:29.119276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:24:28 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0697bbbe44c74c538761737601ceeb10/roles/f239192299174e29a4672c9df068e024 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:24:31.048525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059978e-096f-4458-be93-8a24293ce8b3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.049117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059978e-096f-4458-be93-8a24293ce8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.049117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059978e-096f-4458-be93-8a24293ce8b3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.050697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:31.055771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.056232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.056335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.076856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:31.078073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:31.153820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:31.419862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059978e-096f-4458-be93-8a24293ce8b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:31.448327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:31.545334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059978e-096f-4458-be93-8a24293ce8b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:31.585670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:31.586936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.587297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.587731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.598864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:31.604400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:31.605827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:24:31.607032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:31.608999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 29 21:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:31.611606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:31.667543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:31.741826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0980-e065-44b2-a148-0070ac2218da None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:31.788011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:31.788797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.789026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.789331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.790012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:31.790313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:31.791687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:31.800572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 29 21:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:31.806646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:31.924308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:31.949935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5e7c-1e7a-445b-a611-17337ad05209 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:31.993210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:31.994107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.994549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.995019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.006840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.009053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:32.025726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:32.034279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:32.035205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:32.035205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:32.035205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:32.035205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:32.035643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:32.189527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79faab3-5af3-4f45-b141-185113a99c77 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:32.202419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:32.241742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:32.242361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:32.243149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:32.243489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.244311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:32.244508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.246028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:32.264418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:32.265729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:32.390826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601ff5-7c2b-409d-bb0d-3eda63d4b0c7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:32.437899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:32.438717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:32.439038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:32.439320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.442653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:32.451402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.452972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:32.466367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:32.467422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:32.468397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:32.469179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:32.473335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:32.607412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14080e8-e28f-41e2-af4f-dc7f838066a3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:32.654432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:32.655138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:32.655595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:32.655915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.656970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:32.657726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.659698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:32.683692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:32.689774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:32.806544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3ab31-1cf8-4763-ba1f-dad0d781c12c None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:32.849535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:32.850392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:32.850804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:32.851082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.926279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:32.934156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.935780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:33.065525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:24:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:24:33.073810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:33.110716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:33.208091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a087d-4541-4c0c-976c-f122a949f56a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:33.255098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:33.255714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.255983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.256247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.272058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:33.274173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:33.294489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:33.295906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:33.297159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:33.297159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:33.297159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:33.300289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:33.301703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:33.413465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:33.424208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7244b-5c74-40e3-8db0-59c97dd93d39 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:33.470993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:33.471612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.471879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.472161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.472909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:33.473133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:33.474510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:33.487565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:33.494934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:33.596973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:33.627755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4116ec0-0177-48b8-86c4-f312764c5faf None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:33.666125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:33.666686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.666967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.667244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.674852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:33.676343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:33.688380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:33.689515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:33.690452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:33.695590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:33.821277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5e1b0-9106-494d-af27-d8e7cc7f6e32 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:33.867732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:33.868357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.868704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.869001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.869666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:33.869970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:33.871387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099fa8-2c6f-4e89-a03c-e8f91231fd31 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:33.884579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:33.891805 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:34.007051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeac66-1e3d-465c-8ae6-19f2b68af5e3 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:34.040726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:34.041326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:34.041602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:34.041874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:34.042598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] RBAC: Adding query filter params (role.id=f239192299174e29a4672c9df068e024, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0697bbbe44c74c538761737601ceeb10) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:34.042903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:34.044328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e7f19-da8c-4d9a-b252-2ddc1a1057c5 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:34.050159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3/role_assignments?user.id=0697bbbe44c74c538761737601ceeb10&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=f239192299174e29a4672c9df068e024 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:36.138601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34a-afc4-41bf-8f1f-6f195945ee0e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:36.139206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34a-afc4-41bf-8f1f-6f195945ee0e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:36.139354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8f34a-afc4-41bf-8f1f-6f195945ee0e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:36.141710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:36.147240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:36.147759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:36.147859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:36.155518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:36.169064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:36.170432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:36.255352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:36.528813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f34a-afc4-41bf-8f1f-6f195945ee0e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:36.552720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:36.664079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8f34a-afc4-41bf-8f1f-6f195945ee0e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:36.706859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:36.707633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:36.707889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:36.708195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:36.719871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:36.724800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:36.726304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:36.727340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:36.735196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:36.855201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc771f5-2c8e-46af-968d-9c60784001fe None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:36.905219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:36.911188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:36.911873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:36.912149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:36.912403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:36.913152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:36.913440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:36.915440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:36.924743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:36.931519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.027119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:37.053336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25712bd9-34d4-4170-9e65-0c5b1e41cf38 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.094841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.095479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.095757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.096023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.107915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.109509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:37.126819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:37.128155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:37.129272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:37.129272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:37.129272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:37.129272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:37.139464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.268080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2c899-f33c-4a74-be2b-a7433342313a None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.313382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.314410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.314908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.315327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.316469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:37.317051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.319473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:37.336697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:37.343263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.464734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842b83d-6a8b-40ca-bec5-513aaabe276b None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.496490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:37.505841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.506487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.506748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.507019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.517220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.518851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:37.531796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:37.537892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:37.539687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:37.540478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.609977 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:37.677286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e26b2-6e79-422a-8080-c618830931dd None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.718656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.719422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.719781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.720203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.721065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:37.721516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.723053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:37.747165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:37.752873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.870464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195b3d-a603-4949-a528-027f2abaaa67 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.911322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.912058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.912384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.912687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.979284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:37.992890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.994294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:38.117449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:24:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:24:38.123750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:38.244308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947db8da-d336-4638-953f-ff9e5433bee8 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:38.259175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:38.286644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:38.287568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:38.287568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:38.287785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:38.299939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:38.301504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:38.317532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:38.319304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:38.320640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:38.320640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:38.320640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:38.320640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:38.324853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:38.428132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:38.442719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405111c8-ee0a-4b2c-9b0a-19a1927adef5 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:38.485074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:38.485726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:38.486038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:38.486442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:38.487121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:38.487543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:38.489349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:38.503238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:38.509415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:38.633837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dfadd-ef3b-47e1-8151-b9185eb9193b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:38.671656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:38.672285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:38.672605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:38.672854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:38.679650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:38.681265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:38.692426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:38.693529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:38.694468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:38.700695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:38.816318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48788-64be-460f-bd40-81eb39b07fc9 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:38.858622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:38.859280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:38.859775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:38.860161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:38.861137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:38.861474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:38.863160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0d1de-dc49-42b8-b499-aad541afb49e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:38.878717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:38.885469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:38.920433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:39.009584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495de1-da58-4745-90ab-79e04f9ee60f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:39.052271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:39.052975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:39.053325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:39.053630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:39.054495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0697bbbe44c74c538761737601ceeb10) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:39.055504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:39.057153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006dcfa-1011-4e03-b2b8-468a6bff4d06 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:39.065038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:24:38 2022] GET /identity/v3/role_assignments?user.id=0697bbbe44c74c538761737601ceeb10&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:40.982597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98371e-fa77-4e52-9f6f-f8ae153aef1a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:40.983152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98371e-fa77-4e52-9f6f-f8ae153aef1a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:40.983152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98371e-fa77-4e52-9f6f-f8ae153aef1a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:40.984495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:40.991285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:40.991708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:40.991819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:41.012323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:41.013648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:41.082426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:41.359283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98371e-fa77-4e52-9f6f-f8ae153aef1a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:41.372168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:41.480107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98371e-fa77-4e52-9f6f-f8ae153aef1a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:41.519337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:41.520212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:41.520331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:41.520514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:41.531631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:41.536806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:41.541164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:41.542780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:24:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:41.547138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:41.568211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:41.666826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e713dd-dd60-46c0-aebc-411a8cd78dee None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:41.704903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:41.705514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:41.705794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:41.706061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:41.706729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:41.707030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:41.708409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:41.717774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:41.723861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:41.795083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:41.828213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207774b-4096-40fb-bf6b-96657aeafedc None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:41.862877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:41.863507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:41.863786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:41.864059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:41.874464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:41.875919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:41.891143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:41.892182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:41.893415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:41.893415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:41.893415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:41.893415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:41.898714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:41.985028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:42.004454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665e439-d5ae-4131-a8f3-10a7571d22ca None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.038320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.038920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.039195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.039461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.040089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:42.040386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.041774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.053269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:42.058996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.162650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10022b7f-5455-4f60-9fef-6119efb03b0f None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.186640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:42.204262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.204940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.205218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.205489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.214214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.215815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:42.227963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:42.229262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:42.230234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:42.234932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.337800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a4d9c-de75-4c35-9d00-0459eab1b792 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.372856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.373498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.373749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.374045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.374869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:42.375178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.376633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.385385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:42.394110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:42.400450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.507195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f81d87-1c49-4be7-a5b1-df476e0649fd None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.541238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.541863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.542141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.542413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.553435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.554870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:42.568647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:42.569774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:42.570840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:42.570840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:42.570840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:42.570840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:42.575974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.674603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3dcd05-d671-47a6-8b45-58ccd35b62d6 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.708022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.708638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.708913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.709181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.709809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:42.710109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.711990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.722902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:42.731217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.763326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:42.837429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21d81-478f-4538-bf3a-cc41b095af33 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:42.871890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:42.872529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:42.872836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:42.873107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:42.879757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:42.881216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:42.893079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:42.894158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:42.895086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:42.901226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:42.962251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:43.004548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe71c2-4ffa-4a8d-9f14-1ae5ce0a9092 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:43.039407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:43.040001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:43.040281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:43.040554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:43.041274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:43.041715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:43.042992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f0fe3-f9e4-40f8-a6a7-32a33ea60213 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:43.057263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:43.063314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:43.153802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:43.162290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a1ce3-9f42-4140-b585-9d462741ef73 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:43.204199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:43.204925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:43.205162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:43.205532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] PATH_INFO: `/v3/projects/daeb37b1a68a454294380a1715813f37/users/0697bbbe44c74c538761737601ceeb10/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:43.207633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] RBAC: Authorizing `identity:create_grant(project_id=daeb37b1a68a454294380a1715813f37, user_id=0697bbbe44c74c538761737601ceeb10, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:43.209330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a1b92-6b06-4da3-bbfc-93fd65edfd61 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:43.225593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 29 21:24:43 2022] PUT /identity/v3/projects/daeb37b1a68a454294380a1715813f37/users/0697bbbe44c74c538761737601ceeb10/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:24:45.109653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905c53-9f3a-4c94-9821-d59e9fc9c8cf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:45.110178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905c53-9f3a-4c94-9821-d59e9fc9c8cf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:45.110320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15905c53-9f3a-4c94-9821-d59e9fc9c8cf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:45.111591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:45.116606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:45.117658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:45.117761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:45.135935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:45.137145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:45.213586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:45.487806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15905c53-9f3a-4c94-9821-d59e9fc9c8cf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:45.589488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15905c53-9f3a-4c94-9821-d59e9fc9c8cf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:45.608721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:45.627910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:45.628533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:45.628837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:45.629107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:45.639609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:24:45.644338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:45.645561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:45.646833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 29 21:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:45.652070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:45.735769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:45.755290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5da073-e222-4016-976f-cf366fe51908 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:45.788540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:45.789655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:45.789927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:45.790192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:45.790846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:45.791142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:45.792488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:45.800418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 29 21:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:45.806793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:45.909538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc7c2-db9a-47a4-917b-21b737fe8a6f None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:45.943980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:45.944627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:45.944913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:45.945214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:45.955246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:45.956722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:45.971279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:45.973147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:45.974087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:45.974087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:45.974087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:45.974087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:45.978939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:46.079051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68722fa9-8c88-42c5-a395-0b26687da2db None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:46.113220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:46.113817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:46.114107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:46.114362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:46.114993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:46.115289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:46.116662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:46.129012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:46.134990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:46.167179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:46.251210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425da3db-077c-49cf-8912-bc94b98cc05b None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:46.285389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:46.286005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:46.286280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:46.286550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:46.295295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:46.296764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:24:46.308363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:46.309538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:46.310539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:46.315593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:46.319471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:46.416766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d015030-b9c5-4118-9e57-ea58c6bc46c2 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:46.450080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:46.450680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:46.450958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:46.451229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:46.451953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:46.452256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:46.453710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:46.471613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 29 21:24:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:46.480319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:46.573742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:46.584733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463a509-3d08-446b-ae78-30806f44087c None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:46.619972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:46.620570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:46.620874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:46.621142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:46.691407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:46.693288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:46.806138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:46.817452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 29 21:24:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:24:46.823426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:46.888491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:46.929750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d650619-a19b-4faf-9eba-4f6c73d44af7 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:46.965691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:46.966353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:46.966664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:46.966944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:46.977980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:46.979521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:24:46.994482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:46.995624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:24:46.996961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:24:46.996961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:24:46.996961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:46.996961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:24:47.001948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:47.112647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d235450-d328-4ddf-b96e-283cf1373c74 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:47.149714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:47.150377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:47.150621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:47.150954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:47.151625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:47.151926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:47.153705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:47.166511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 29 21:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:24:47.173211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:47.282607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94713be-19d5-4a56-bba7-691f564f5b88 None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:47.296545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:47.321136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:47.322026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:47.322306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:47.322652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:47.330006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:47.331440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:24:47.343583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:47.344804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: ERROR keystone.server.flask.application Aug 29 21:24:47.345975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 29 21:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:47.350789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:47.410727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:47.459603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd91139-f8b4-444e-881e-f2ff87047701 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:47.492652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:47.493257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:47.493529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:47.493800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:47.494481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:47.494782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:47.496205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed532b7-2ced-4efb-adba-c51ab4c20f65 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:47.508276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 29 21:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:47.514374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:47.613501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2471812-9dec-4dbb-828f-d014d2dea85e None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:47.647382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:47.647979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:47.648259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:47.648648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:47.649377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] RBAC: Adding query filter params (role.id=1036ab5f43fc4e2ba01419fa66bbfb11, scope.project.id=daeb37b1a68a454294380a1715813f37, user.id=0697bbbe44c74c538761737601ceeb10) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:47.649688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:47.651137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc30ebb-daf3-476d-a31b-c44e8fb2aa4e None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:47.657218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 29 21:24:47 2022] GET /identity/v3/role_assignments?user.id=0697bbbe44c74c538761737601ceeb10&scope.project.id=daeb37b1a68a454294380a1715813f37&role.id=1036ab5f43fc4e2ba01419fa66bbfb11 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:49.567018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a9dba-2505-4766-be2b-d8610b6056a3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:49.567555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a9dba-2505-4766-be2b-d8610b6056a3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:49.567648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a9dba-2505-4766-be2b-d8610b6056a3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:49.569082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:49.574741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:49.575167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:49.575279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:49.593206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:49.594371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:49.623460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:49.661819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:49.928837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a9dba-2505-4766-be2b-d8610b6056a3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:50.030038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a9dba-2505-4766-be2b-d8610b6056a3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:50.042049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:50.068147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:50.068804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:50.069085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:50.069360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:50.070213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:50.071738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:50.080515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:50.081756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:50.082800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:24:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:24:50.088104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:50.189972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06192432-ee92-4e33-a9a9-d1c1a3720b71 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:50.237554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:50.238422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:50.238913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:50.239270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:50.240112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:50.240521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:50.242565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206c96-a6a1-4ee1-acb1-225f913131a4 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:50.248224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 29 21:24:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:50.254402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:50.352769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36d75b-7a37-49aa-b098-33f164054225 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:50.385443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:50.386033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:50.386310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:50.386580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] PATH_INFO: `/v3/services` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:50.387198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:50.387498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:50.388891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc23256-3b1c-47f1-b073-07018d24dbe6 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:50.393190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 29 21:24:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:52.279966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc08d7-f2b2-4272-9b28-4467cadba372 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:52.280482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc08d7-f2b2-4272-9b28-4467cadba372 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:52.280630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc08d7-f2b2-4272-9b28-4467cadba372 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:52.281919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:52.287135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf8c0-59ce-4c27-a49c-77321bdaad8e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:52.287517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf8c0-59ce-4c27-a49c-77321bdaad8e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:52.287680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edf8c0-59ce-4c27-a49c-77321bdaad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:52.305972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c1edf8c0-59ce-4c27-a49c-77321bdaad8e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:52.307147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1edf8c0-59ce-4c27-a49c-77321bdaad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:52.336510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:52.374254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:52.656305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc08d7-f2b2-4272-9b28-4467cadba372 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:52.687475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:52.761316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc08d7-f2b2-4272-9b28-4467cadba372 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:52.796766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:52.797393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:52.797673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:52.797948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:52.798634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:52.800028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd22b65-0541-4b23-8fa5-495937680f4a None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:52.819143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 29 21:24:52 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:24:54.777223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7988-f51e-4d6f-b331-ba25f4cf7104 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:54.777799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7988-f51e-4d6f-b331-ba25f4cf7104 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:54.777799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7988-f51e-4d6f-b331-ba25f4cf7104 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:54.779104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:54.784345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:54.784771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:54.784946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:54.802958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:54.804168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:54.877453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:55.149411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7988-f51e-4d6f-b331-ba25f4cf7104 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:55.265009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7988-f51e-4d6f-b331-ba25f4cf7104 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:55.301148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:55.301774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:55.302049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:55.302319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:55.302896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:55.304253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:55.312906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:55.314073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:55.314932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:24:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:55.320297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:55.338713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:55.424562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d3fc8-c1e9-4bf1-85b2-21706d81a973 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:55.459564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:55.460711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:55.461002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:55.461279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:55.461902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:55.462207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:55.463572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:55.468943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 29 21:24:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:55.475040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:55.510974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:55.584251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c75e2-6ff4-4a15-983b-1fb34c55fb38 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:55.623636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:55.624233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:55.624508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:55.624819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:55.625512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] RBAC: Adding query filter params (interface=public, service_id=8278a4e904fa4b378821b1dd187f4a9b, region_id=RegionOne) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:55.625810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:55.627174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b068-9d96-4345-a2b8-e41505f2b669 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:55.632733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 29 21:24:55 2022] GET /identity/v3/endpoints?service_id=8278a4e904fa4b378821b1dd187f4a9b&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:55.638436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:55.722615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:55.741137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad37ec-5c0e-4938-8169-ccbb35dfd3fb None admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:55.775141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:55.775736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:55.776014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:55.776286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:55.777167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:55.778526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e264ac6-386d-4026-a62c-bb0cc453b70c None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:55.785389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 29 21:24:55 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:24:57.685220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064e0bd-ac1c-4445-adb8-f7d769ba5ac7 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:57.685854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064e0bd-ac1c-4445-adb8-f7d769ba5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:57.685854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064e0bd-ac1c-4445-adb8-f7d769ba5ac7 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:57.687118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:57.692392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:57.692850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:57.692962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:57.711477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:57.713013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:57.779757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:24:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:24:58.045565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064e0bd-ac1c-4445-adb8-f7d769ba5ac7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:58.142633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:58.149612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064e0bd-ac1c-4445-adb8-f7d769ba5ac7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:58.186511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:58.187437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:58.187868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:58.188289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:58.189252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:58.191472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:24:58.201592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:24:58.203989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application with Lock( Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: ERROR keystone.server.flask.application Aug 29 21:24:58.205079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 29 21:24:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:24:58.209032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:58.288130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:58.311843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69639f0e-2cae-43f0-8540-34769ddf7bbc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:58.345805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:58.346412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:58.346690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:58.346962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] PATH_INFO: `/v3/services` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:58.347586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:24:58.347885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:58.349262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80730f5-2c7a-413f-8d49-c4a1511087d2 None admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:58.355104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 29 21:24:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:58.360870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:58.461969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2750ae-5f58-4a96-b37b-39dc8609ff34 None admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:58.495350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:58.495945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:58.496220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:58.496486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:58.497174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:58.498534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6454ee-f384-4198-ad0a-c373dc70232a None admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:58.518247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:24:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:25:08.352083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=30 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:25:08.513770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:04.307732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af57b-db0e-4fbf-b369-06035e320559 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:04.308315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af57b-db0e-4fbf-b369-06035e320559 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:04.308315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af57b-db0e-4fbf-b369-06035e320559 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:04.309973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:04.315899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afcca-0744-4c74-9133-258bf06369b5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:04.316350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afcca-0744-4c74-9133-258bf06369b5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:04.317298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621afcca-0744-4c74-9133-258bf06369b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:04.409630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-621afcca-0744-4c74-9133-258bf06369b5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:04.411337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-621afcca-0744-4c74-9133-258bf06369b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:04.545179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:04.865124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:04.865408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:04.865686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:04.867033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 29 21:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:04.873680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97867729-c7d2-411c-9bd9-ec1454eeb8c5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:04.873975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97867729-c7d2-411c-9bd9-ec1454eeb8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:04.874251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97867729-c7d2-411c-9bd9-ec1454eeb8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:04.966377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-97867729-c7d2-411c-9bd9-ec1454eeb8c5 None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:04.967527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-97867729-c7d2-411c-9bd9-ec1454eeb8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:04.990286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:05.052642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 29 21:26:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:05.061804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:05.063573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:05.216453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce89fa5-116c-4d4f-99c2-5d8710cb6841 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:05.301020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:05.301671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:05.301949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:05.302218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:05.303837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:05.382774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:05.452314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:05.454938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72efb6-a6ec-40a8-818a-d8fbb650f49d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:05.484473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:07.712972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5ae71-3254-4980-8f25-fc22af325d97 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:07.713609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5ae71-3254-4980-8f25-fc22af325d97 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:07.713609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5ae71-3254-4980-8f25-fc22af325d97 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:07.714758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:07.721325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e4f4-7e49-4a2c-ba4b-cf9f1b6dab3c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:07.721325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e4f4-7e49-4a2c-ba4b-cf9f1b6dab3c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:07.721325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e4f4-7e49-4a2c-ba4b-cf9f1b6dab3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:07.806546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f8d7e4f4-7e49-4a2c-ba4b-cf9f1b6dab3c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:07.807777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f8d7e4f4-7e49-4a2c-ba4b-cf9f1b6dab3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:07.911687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:08.210348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221841-4719-4eb5-b235-029316701d86 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:08.210785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221841-4719-4eb5-b235-029316701d86 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:08.211087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88221841-4719-4eb5-b235-029316701d86 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:08.212453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 658/1314] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 29 21:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:08.218509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528c78-fb96-4e0b-abe8-cbbf861023bf None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:08.218788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528c78-fb96-4e0b-abe8-cbbf861023bf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:08.219055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528c78-fb96-4e0b-abe8-cbbf861023bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:08.231598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:08.299431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe528c78-fb96-4e0b-abe8-cbbf861023bf None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:08.300686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe528c78-fb96-4e0b-abe8-cbbf861023bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:08.381355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 657/1315] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 29 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:08.390587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88221841-4719-4eb5-b235-029316701d86 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:08.392118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-88221841-4719-4eb5-b235-029316701d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:08.503613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88221841-4719-4eb5-b235-029316701d86 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:08.578606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:08.581297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:08.582003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:08.582477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:08.583048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:08.584922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:08.702843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:08.704317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c620e0-58f5-4ee5-b067-4ed71c5c90d0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:08.725181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 659/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:11.178299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:11.178861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:11.178861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:11.180115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 658/1317] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:11.186535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ad60-66db-41e8-a93e-f8dfc9e1dc18 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:11.186992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ad60-66db-41e8-a93e-f8dfc9e1dc18 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:11.187108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7ad60-66db-41e8-a93e-f8dfc9e1dc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:11.254728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-88e7ad60-66db-41e8-a93e-f8dfc9e1dc18 None None] MFA Rules not processed for user `59fd3c40e3254e4abe88a9801df7949d`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:11.255923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-88e7ad60-66db-41e8-a93e-f8dfc9e1dc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:11.306829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:11.352946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 660/1318] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:11.635919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:11.637518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:11.741836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c369da-2c2d-4692-8487-0edec5b530ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:11.744669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:11.818201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:11.818813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:11.819088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:11.819356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:11.821043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:11.935084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:11.936424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4b823-e4f3-43f0-80a4-2c7820aacf8f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:11.954870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 659/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:14.478331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:14.478914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:14.478914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:14.480254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 661/1320] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:26:14.485867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4cf0a-8134-40da-9592-9936e9d365aa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:14.486288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4cf0a-8134-40da-9592-9936e9d365aa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:14.486411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4cf0a-8134-40da-9592-9936e9d365aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:14.527817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:14.553536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ead4cf0a-8134-40da-9592-9936e9d365aa None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:14.554765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ead4cf0a-8134-40da-9592-9936e9d365aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:14.655748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 660/1321] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:14.943187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:14.944809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:15.048429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7161f7-136c-4339-a00d-9288b54cc8eb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:15.124283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:15.124827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:15.125114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:15.125368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:15.126954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:15.134636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:15.245030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:15.246632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a52643-7ffa-47eb-a1c1-38da471dff9f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:15.267077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 662/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:17.915921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:17.916491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:17.916491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:17.917885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 661/1323] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:17.923025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf778-aa9e-4e97-9491-b0b3c923c941 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:17.923435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf778-aa9e-4e97-9491-b0b3c923c941 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:17.923570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf778-aa9e-4e97-9491-b0b3c923c941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:17.993963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bdabf778-aa9e-4e97-9491-b0b3c923c941 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:18.003141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bdabf778-aa9e-4e97-9491-b0b3c923c941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:18.061912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:18.102579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 663/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:18.391890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:18.393471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:18.511904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068024-b24a-448d-9942-38aebcf9cec5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:18.526027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:18.595701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:18.596310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:18.596600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:18.596878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:18.599585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:18.716974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:18.718304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7a1c08-5102-443f-a9f6-614f8a8ab8f7 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:18.737838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 662/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:20.782935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:20.783701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:20.783701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:20.784901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 664/1326] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:20.790388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356f66-d44c-4dee-9336-709c22bcda9f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:20.790862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356f66-d44c-4dee-9336-709c22bcda9f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:20.790964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0356f66-d44c-4dee-9336-709c22bcda9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:20.846145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:20.877992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a0356f66-d44c-4dee-9336-709c22bcda9f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:20.879874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a0356f66-d44c-4dee-9336-709c22bcda9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:20.981226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 663/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:21.269068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:21.270622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:21.383648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ca37-8da2-4789-aec1-1e972d1eaf82 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:21.467376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:21.467978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:21.468253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:21.468520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:21.470160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:21.488889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:21.600132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:21.601618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e734691-2b92-402c-9031-14ed43bb6a7f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:21.620124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 665/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:23.621156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:23.621831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:23.621831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:23.623061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 664/1329] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:23.629461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31747d-59a9-447e-ab30-f7ec173082a6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:23.630066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31747d-59a9-447e-ab30-f7ec173082a6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:23.630066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31747d-59a9-447e-ab30-f7ec173082a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:23.716932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ca31747d-59a9-447e-ab30-f7ec173082a6 None None] MFA Rules not processed for user `59fd3c40e3254e4abe88a9801df7949d`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:23.718209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ca31747d-59a9-447e-ab30-f7ec173082a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:23.789970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:23.825004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 666/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:24.112641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:24.114184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:24.238735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f8f07-2aff-47bf-9715-b35ed8bb7cda None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:24.258513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:24.322998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:24.323603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:24.323879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:24.324150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:24.325808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:24.441142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:24.442497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856cf79-a12a-4006-aa1e-565951e19551 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:24.461722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 665/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:26.663768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.664311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.664311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.665732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 667/1332] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:26.671603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385532-308e-4c37-8836-dbcbe0857974 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.672067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385532-308e-4c37-8836-dbcbe0857974 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.672183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3385532-308e-4c37-8836-dbcbe0857974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.741105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:26.743945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e3385532-308e-4c37-8836-dbcbe0857974 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:26.745926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e3385532-308e-4c37-8836-dbcbe0857974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:26.848469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 666/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:27.151001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.152559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:27.258392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96a05-67cd-4766-aad8-cd83833a8cf5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.331620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.332219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.332497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.332791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.334389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:27.363898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:27.464499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.466180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83404f-af7e-4b5d-9f31-9bf58d12948c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.485192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 668/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:29.479162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.479760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.479760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.481374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 667/1335] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:29.486919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a6e4-e6f8-48b9-ba81-79fdf1987c0d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.487395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a6e4-e6f8-48b9-ba81-79fdf1987c0d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.487535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a6e4-e6f8-48b9-ba81-79fdf1987c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.566787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8a64a6e4-e6f8-48b9-ba81-79fdf1987c0d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.568195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8a64a6e4-e6f8-48b9-ba81-79fdf1987c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.650983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:29.673723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 669/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.962647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.964225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:30.067649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b450c8b-8933-4447-b704-5048bc839881 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.094919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:30.140482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.141110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.141388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.141658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.143263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:30.253272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.255340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c8756-f9cf-41db-8d89-2028b4edec28 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.272912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 668/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:32.497959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:32.498497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:32.498497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:32.499858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:32.505023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc496-2438-4c85-8ab2-7f1ccffbbaab None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:32.505466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc496-2438-4c85-8ab2-7f1ccffbbaab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:32.505561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dc496-2438-4c85-8ab2-7f1ccffbbaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:32.570228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-612dc496-2438-4c85-8ab2-7f1ccffbbaab None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:32.571449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-612dc496-2438-4c85-8ab2-7f1ccffbbaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:32.579581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:32.667161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:32.946859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:32.948448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:33.054139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a08a68-7749-4205-9c1a-6f770d0c9ddd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:33.132030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:33.132664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:33.132946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:33.133218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:33.134828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:33.170370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:33.247732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:33.249241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dacad-ac70-4c21-bcf3-d186b1d03b85 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:33.268225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:37.490099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:37.490566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:37.490690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:37.492093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:37.497565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe269-6347-4262-8979-15276870f9f2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:37.498036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe269-6347-4262-8979-15276870f9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:37.498129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abe269-6347-4262-8979-15276870f9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:37.566439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e0abe269-6347-4262-8979-15276870f9f2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:37.567647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e0abe269-6347-4262-8979-15276870f9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:37.659450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:37.671434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:37.961697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:37.963229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:38.068315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70ef5-2848-477e-a0de-98f4f84c6fac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:38.106624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:38.144649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:38.145271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:38.145550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:38.145819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:38.147429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:38.259443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:38.260853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990df7b1-672c-470c-b46e-c61c4d454211 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:38.278678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:40.854104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678380-1a66-474e-9ccd-be179973036f None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.854646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678380-1a66-474e-9ccd-be179973036f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.854646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30678380-1a66-474e-9ccd-be179973036f None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.856406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:40.862425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aa59-0768-4bc2-9832-09b1112b0995 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.863018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aa59-0768-4bc2-9832-09b1112b0995 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.863018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7aa59-0768-4bc2-9832-09b1112b0995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.936703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-76e7aa59-0768-4bc2-9832-09b1112b0995 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:40.937929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-76e7aa59-0768-4bc2-9832-09b1112b0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:40.957798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:41.041296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:41.329923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678380-1a66-474e-9ccd-be179973036f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:41.332259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30678380-1a66-474e-9ccd-be179973036f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:41.442799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30678380-1a66-474e-9ccd-be179973036f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:41.518827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:41.519482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:41.519733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:41.520004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:41.521663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:41.570008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:41.639471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:41.640914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0dfee-6381-45fa-99ce-56e73cd50cc3 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:41.660224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:45.588931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.589426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.589516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.590851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:45.597252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfa826-58bb-4789-8a9a-02347942c25a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.597772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfa826-58bb-4789-8a9a-02347942c25a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.597772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfa826-58bb-4789-8a9a-02347942c25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.671247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-febfa826-58bb-4789-8a9a-02347942c25a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:45.672662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-febfa826-58bb-4789-8a9a-02347942c25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.774611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:45.777677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:46.067816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.069390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.181934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02a4e9-a59c-4d6e-92b8-fbffd57429fe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.232722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:46.260106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.260744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.261027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.261300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.262907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.370406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:46.371870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0c826-5ef1-4c7c-a894-7da8deae29c8 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:46.390228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:51.283010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb952-5d91-4260-ae57-ace0a722270e None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.283583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb952-5d91-4260-ae57-ace0a722270e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.283783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb952-5d91-4260-ae57-ace0a722270e None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.285503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:51.293084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3ab5-e62d-4482-b5bf-384a9b8ed1ea None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.293621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3ab5-e62d-4482-b5bf-384a9b8ed1ea None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.293621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3ab5-e62d-4482-b5bf-384a9b8ed1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.396657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c2d3ab5-e62d-4482-b5bf-384a9b8ed1ea None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:51.396657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c2d3ab5-e62d-4482-b5bf-384a9b8ed1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.418839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:51.610830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:55.891489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6318d5-3fb1-4fa4-a42d-22436dd610f2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.892996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6318d5-3fb1-4fa4-a42d-22436dd610f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.893517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6318d5-3fb1-4fa4-a42d-22436dd610f2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.895443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:26:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:55.901476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f0d3-0604-41b5-9b7d-569ee71fed4b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.901927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f0d3-0604-41b5-9b7d-569ee71fed4b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.902010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f0d3-0604-41b5-9b7d-569ee71fed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.983301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2653f0d3-0604-41b5-9b7d-569ee71fed4b None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:55.984502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2653f0d3-0604-41b5-9b7d-569ee71fed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:56.087697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:56.151247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:57.553956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d5ab-990e-4b24-80dc-396bafc89be0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.554480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d5ab-990e-4b24-80dc-396bafc89be0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.554679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90d5ab-990e-4b24-80dc-396bafc89be0 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.556191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:57.560969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a0fb-0128-4814-a7ec-444a050d71d7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.561376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a0fb-0128-4814-a7ec-444a050d71d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.561761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185a0fb-0128-4814-a7ec-444a050d71d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.641752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e185a0fb-0128-4814-a7ec-444a050d71d7 None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:57.643542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e185a0fb-0128-4814-a7ec-444a050d71d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:57.757267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:01.042339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b751ec-eb99-48a1-9ea3-5e8cc2350937 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.042867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b751ec-eb99-48a1-9ea3-5e8cc2350937 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.042867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b751ec-eb99-48a1-9ea3-5e8cc2350937 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.044299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:01.049466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81277-5574-4773-b007-0d43374fe7fe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.049895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81277-5574-4773-b007-0d43374fe7fe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.050037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81277-5574-4773-b007-0d43374fe7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.086187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:01.131364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0de81277-5574-4773-b007-0d43374fe7fe None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:01.132802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0de81277-5574-4773-b007-0d43374fe7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:01.246768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:01.561548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.562123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.562599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.565344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 29 21:27:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:01.572444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5da93-2213-4774-8f82-092cccc4e3f7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.573022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5da93-2213-4774-8f82-092cccc4e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.573519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5da93-2213-4774-8f82-092cccc4e3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.659711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-31f5da93-2213-4774-8f82-092cccc4e3f7 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:01.661558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31f5da93-2213-4774-8f82-092cccc4e3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:01.717099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:01.792192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 29 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:01.802127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:01.803696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:01.958193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e585cd-e168-419e-b21f-0ee679f62581 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.046168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:02.091682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.092384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.092760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.093094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.094799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:02.199670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.201687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e4f11-7b65-438f-9438-41b0e0aff101 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.222008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:04.415466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.416098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.416098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.417607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:04.424077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cbe4-5876-4d77-8938-00d7066d114c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.425056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cbe4-5876-4d77-8938-00d7066d114c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.425330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688cbe4-5876-4d77-8938-00d7066d114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.506731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5688cbe4-5876-4d77-8938-00d7066d114c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:04.508033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5688cbe4-5876-4d77-8938-00d7066d114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.550665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:04.616681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:04.921741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:04.923942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:05.123505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c8aa1-c895-4557-a252-ba0cfd74e364 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:05.135152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:05.283379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:05.284297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:05.284297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:05.284742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:05.286623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:05.403625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:05.405704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95437153-cdc3-4af7-9bc5-c0440a99ee61 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:05.427188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:07.730646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194865-b41f-4377-8015-292613c1d0f3 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:07.731541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194865-b41f-4377-8015-292613c1d0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:07.731825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194865-b41f-4377-8015-292613c1d0f3 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:07.733251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:07.739388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d896-1942-459d-80a2-7974aced34ec None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:07.740239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d896-1942-459d-80a2-7974aced34ec None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:07.740517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d896-1942-459d-80a2-7974aced34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:07.757804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:07.816032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ed76d896-1942-459d-80a2-7974aced34ec None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:07.817441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed76d896-1942-459d-80a2-7974aced34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:07.926794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:08.227035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:08.227681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:08.228163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:08.229740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 29 21:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:08.238263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f6c2b-d1b9-4a76-872b-3e39245209ab None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:08.238615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f6c2b-d1b9-4a76-872b-3e39245209ab None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:08.239038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f6c2b-d1b9-4a76-872b-3e39245209ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:08.321320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-553f6c2b-d1b9-4a76-872b-3e39245209ab None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:08.322543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-553f6c2b-d1b9-4a76-872b-3e39245209ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:08.359433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:08.450408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 29 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:08.458849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:08.460378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:08.622577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5470fe-ae73-425c-9640-ff4044b4ed64 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:08.647852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:08.761139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:08.762029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:08.762507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:08.762895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:08.764600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:08.862038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:08.866758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7baf1-ac14-44a2-84d2-4ea031c03e87 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:08.885526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:09.491295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205ac9-27f3-4840-ad2f-e6af4082c75b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:09.491862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205ac9-27f3-4840-ad2f-e6af4082c75b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:09.492457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205ac9-27f3-4840-ad2f-e6af4082c75b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:09.494234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:09.499666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77bf93-ac0e-47f8-9b19-c6e0feadd85d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:09.499952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77bf93-ac0e-47f8-9b19-c6e0feadd85d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:09.500226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77bf93-ac0e-47f8-9b19-c6e0feadd85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:09.536535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:09.575058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8e77bf93-ac0e-47f8-9b19-c6e0feadd85d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:09.576251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8e77bf93-ac0e-47f8-9b19-c6e0feadd85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:09.689656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:10.284108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.284108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.284108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.287292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 685/1371] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 29 21:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:10.290212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f23f-4492-4b38-a1f2-258139757d52 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.290721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f23f-4492-4b38-a1f2-258139757d52 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.290872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633f23f-4492-4b38-a1f2-258139757d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.372920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3633f23f-4492-4b38-a1f2-258139757d52 None None] MFA Rules not processed for user `00f82e19c26b45d2a4ac373665f8c9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:10.374141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3633f23f-4492-4b38-a1f2-258139757d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.428380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:10.461551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 687/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:10.468997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:10.471324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.582258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5979b-577e-4486-aee8-6dcf260660e7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:10.629440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:10.667905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:10.668334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.668676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.668917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.670534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.830515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:10.832366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68ae22-f29c-4d32-b5c8-0dbfa9062e0a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:10.859610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 686/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:10.931245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.931730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.931730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.933305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:10.941202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e910-8197-489e-8109-59b0e1eae0b0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.941202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e910-8197-489e-8109-59b0e1eae0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.941202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747e910-8197-489e-8109-59b0e1eae0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.032181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f747e910-8197-489e-8109-59b0e1eae0b0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:11.034235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f747e910-8197-489e-8109-59b0e1eae0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.056897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:11.137703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:11.428948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.430532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.580254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db43d6-d2bf-4955-9a72-5d0b2dc01d89 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.634698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:11.700392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.700941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.701216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.701484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.703073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.794564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.796289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091bd80-04de-442a-b241-cc92535a9ac1 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.814505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:13.801651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:13.802183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:13.802183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:13.803561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:13.809858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebac44-bd41-4708-9c95-ee4a796ecd3b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:13.810321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebac44-bd41-4708-9c95-ee4a796ecd3b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:13.810419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebac44-bd41-4708-9c95-ee4a796ecd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:13.903579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3cebac44-bd41-4708-9c95-ee4a796ecd3b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:13.904998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3cebac44-bd41-4708-9c95-ee4a796ecd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:13.915144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:14.009057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:14.300081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:14.301650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:14.490260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b949-de0a-48ff-a4f7-a8447e97ed92 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:14.494552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:14.634246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:14.635085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:14.635468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:14.636013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:14.638178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:14.743506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:14.745170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd438e-46d5-45fc-8594-553a15122f20 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:14.775128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:16.846610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:16.847198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:16.847198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:16.848651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:16.853942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa448c-7844-4834-966b-5c4b87eaeca6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:16.854424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa448c-7844-4834-966b-5c4b87eaeca6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:16.854512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa448c-7844-4834-966b-5c4b87eaeca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:16.869457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:16.934894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-78aa448c-7844-4834-966b-5c4b87eaeca6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:16.936036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-78aa448c-7844-4834-966b-5c4b87eaeca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.052623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:17.340862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.342393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.508497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:17.522949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17cac-0a5e-4ed5-bddd-64595db28905 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.649763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.650313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.650588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.650858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.652686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.760404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.762093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54edf64c-c87b-45a7-baa5-cb546317b1d3 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.779609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:17.791787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:20.078490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.079165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.079165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.080577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:20.086822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d0dde-bae2-46f6-867f-ff835bef8249 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.087500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d0dde-bae2-46f6-867f-ff835bef8249 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.087500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d0dde-bae2-46f6-867f-ff835bef8249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.171019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f78d0dde-bae2-46f6-867f-ff835bef8249 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:20.172165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f78d0dde-bae2-46f6-867f-ff835bef8249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.290218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:20.631635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:20.633848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.748740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:20.827498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec032b01-28ea-40af-9565-4a68c7e3bd50 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:21.008378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:21.008964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:21.009255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:21.009540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:21.011248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:21.078688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:21.128431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:21.130269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d759ab-7a31-499a-ae42-b1fdbb8a97e0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:21.157250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:23.566886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f662a-5388-47f7-be4f-328904e30169 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.567462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f662a-5388-47f7-be4f-328904e30169 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.567713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f662a-5388-47f7-be4f-328904e30169 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.569471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:23.576752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1388-0f25-4cca-ac91-79c8b8bcf0da None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.577412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1388-0f25-4cca-ac91-79c8b8bcf0da None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.577685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1388-0f25-4cca-ac91-79c8b8bcf0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.676681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-015c1388-0f25-4cca-ac91-79c8b8bcf0da None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:23.678191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-015c1388-0f25-4cca-ac91-79c8b8bcf0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:23.819808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:24.131860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f662a-5388-47f7-be4f-328904e30169 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.133493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-387f662a-5388-47f7-be4f-328904e30169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:24.238159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:24.319315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f662a-5388-47f7-be4f-328904e30169 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.460069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.460974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.461449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.461930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.464838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:24.516456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:24.580497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.582179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f3812-e0aa-4bba-bc40-335553c37df2 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.603076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:26.125696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220903-cc55-4fd4-b6d6-3bb51fef8803 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.125696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220903-cc55-4fd4-b6d6-3bb51fef8803 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.125696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00220903-cc55-4fd4-b6d6-3bb51fef8803 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.127395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:27:26.133172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c9cec-0120-48b6-9ebc-2ef54aed00ed None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.133646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c9cec-0120-48b6-9ebc-2ef54aed00ed None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.133759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c9cec-0120-48b6-9ebc-2ef54aed00ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.206797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-330c9cec-0120-48b6-9ebc-2ef54aed00ed None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:26.208036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-330c9cec-0120-48b6-9ebc-2ef54aed00ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.325064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:26.578013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c4e7-43fc-43b5-a2e4-098bda107c76 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.578291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c4e7-43fc-43b5-a2e4-098bda107c76 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.578570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1c4e7-43fc-43b5-a2e4-098bda107c76 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.580603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 695/1391] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:26.586814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.587089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.587365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.603874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:26.641203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13794-18e0-46bd-83f4-56f5452fb22a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.641467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13794-18e0-46bd-83f4-56f5452fb22a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.641597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13794-18e0-46bd-83f4-56f5452fb22a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.643132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 696/1392] 10.222.0.20 () {54 vars in 967 bytes} [Mon Aug 29 21:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:27:26.648294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.648997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.649261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.669761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:26.671508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.687942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:26.738853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] MFA Rules not processed for user `0abda3580f52421d816eac7bee95b492`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:26.740099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.804616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 697/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:26.838847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 697/1394] 10.222.0.20 () {58 vars in 1064 bytes} [Mon Aug 29 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:26.846667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:26.848951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.976036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8cc6-de00-45fd-903b-f8d8bad5680a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:26.987980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:27.075118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:27.076065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.076655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.077050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.079668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.089616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:27.090891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.219129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:27.221256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ecfd7-4653-4abb-b31b-2ad863b0352a service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:27.231581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026683c3-9ded-4cb9-a59c-9aa3f043bd6a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:27.235421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:27.242127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 698/1395] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:27.350178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:27.350685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.350962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.351233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.352877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.452374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:27.454679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce345f-799d-4b15-bce8-ca39cc7a54a8 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:27.476651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 698/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:28.011988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5eee-32db-47ed-8d44-032f8f6acf43 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.012276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5eee-32db-47ed-8d44-032f8f6acf43 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.012549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d5eee-32db-47ed-8d44-032f8f6acf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.063636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:28.088097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a60d5eee-32db-47ed-8d44-032f8f6acf43 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:28.089392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a60d5eee-32db-47ed-8d44-032f8f6acf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:28.187943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 699/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:28.236343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.236873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.237269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.240657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 699/1398] 10.222.0.20 () {54 vars in 943 bytes} [Mon Aug 29 21:27:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:28.258336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edd1f-4d07-4579-9bc4-5f600f2e9831 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.259559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edd1f-4d07-4579-9bc4-5f600f2e9831 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.259847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edd1f-4d07-4579-9bc4-5f600f2e9831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.355432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ec9edd1f-4d07-4579-9bc4-5f600f2e9831 None None] MFA Rules not processed for user `06c36d7a8a2246a38fe16e32a4607aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:28.356694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec9edd1f-4d07-4579-9bc4-5f600f2e9831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:28.430549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:28.481530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 700/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Mon Aug 29 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:28.491291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:28.492870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:28.504035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:28.652437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccd600-7c45-48b3-9e48-8ee75dad4610 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:28.757480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:28.764852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:28.765472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.765750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.766024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.767722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:28.855761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:28.857431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec6eb2-b07b-4d83-985f-971ab8c0bd31 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:28.867009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 700/1400] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:27:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:27:29.501633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:29.502162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:29.502162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:29.503543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 701/1401] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:29.509351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7f66-184c-4dac-8fe4-a1bdbf6da4c6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:29.509834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7f66-184c-4dac-8fe4-a1bdbf6da4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:29.509927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc7f66-184c-4dac-8fe4-a1bdbf6da4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:29.592343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a1cc7f66-184c-4dac-8fe4-a1bdbf6da4c6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:29.593596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a1cc7f66-184c-4dac-8fe4-a1bdbf6da4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:29.681269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:29.706079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 701/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:30.008369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:30.009941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:30.180459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597eba0-e451-46c0-9731-ae2fbc65a5c9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:30.239054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:30.331226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:30.331837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.332113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.332382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.334035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:30.444697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:30.448681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8801f-0524-4287-a264-ba3d85334dee service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:30.468464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 702/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:32.511912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.512524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.512793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.514569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 702/1404] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:32.520310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1669966-a53f-443a-8629-be7acf554650 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.520844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1669966-a53f-443a-8629-be7acf554650 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.520948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1669966-a53f-443a-8629-be7acf554650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.594293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:32.605324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b1669966-a53f-443a-8629-be7acf554650 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:32.606557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b1669966-a53f-443a-8629-be7acf554650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:32.719252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 703/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:33.017986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:33.020855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:33.208010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46449-a06a-48e9-b52a-8780aef3cff0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:33.288204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:33.343969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:33.344576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:33.344946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:33.345213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:33.346790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:33.456721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:33.459283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12877fb9-65b5-40fc-92ad-2c535b11b2f5 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:33.478172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 703/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:35.499704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:35.500304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:35.500304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:35.501934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 704/1407] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:35.507871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e294-85f6-459d-9d7b-9f3dea035e79 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:35.509143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e294-85f6-459d-9d7b-9f3dea035e79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:35.509432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e294-85f6-459d-9d7b-9f3dea035e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:35.593281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3e09e294-85f6-459d-9d7b-9f3dea035e79 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:35.595953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e09e294-85f6-459d-9d7b-9f3dea035e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:35.630007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:35.740672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 704/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:36.056170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:36.057909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:36.206742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:36.238039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94047af3-de5e-485a-91f8-aa6e9c7cc452 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:36.381648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:36.382242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:36.382550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:36.382848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:36.384711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:36.550544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:36.563490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:36.566924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6dfecf-eb5b-4ec1-b811-cede1f300279 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:36.589502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 705/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:45.735083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=33 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:46.586629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:51.217873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a3ae-196d-4614-b6a2-897f18d58dc0 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:51.218433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a3ae-196d-4614-b6a2-897f18d58dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:51.218433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9a3ae-196d-4614-b6a2-897f18d58dc0 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:51.220288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 705/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 29 21:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:51.225679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa517-81b7-4f25-9500-abeb117a1f4d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:51.226125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa517-81b7-4f25-9500-abeb117a1f4d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:51.226231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa517-81b7-4f25-9500-abeb117a1f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:51.302719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ae7fa517-81b7-4f25-9500-abeb117a1f4d None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:51.303937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae7fa517-81b7-4f25-9500-abeb117a1f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:51.408788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 706/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 29 21:27:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:51.423426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61007-a3a4-4247-af63-7bedb562993f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:51.423653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61007-a3a4-4247-af63-7bedb562993f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:51.423925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec61007-a3a4-4247-af63-7bedb562993f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:51.425312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 706/1412] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 29 21:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:51.429872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60744e-7252-4cfb-92b3-a17e6485c86b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:51.430161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60744e-7252-4cfb-92b3-a17e6485c86b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:51.430438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60744e-7252-4cfb-92b3-a17e6485c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:51.503507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:51.506779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7c60744e-7252-4cfb-92b3-a17e6485c86b None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:51.508568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7c60744e-7252-4cfb-92b3-a17e6485c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:51.622595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 707/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 29 21:27:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:53.995888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:53.996475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:53.996475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:53.997829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:54.003746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2442b6-d94f-45f4-9baa-c43dc0a0fc34 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:54.004182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2442b6-d94f-45f4-9baa-c43dc0a0fc34 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:54.004297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2442b6-d94f-45f4-9baa-c43dc0a0fc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:54.073129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fc2442b6-d94f-45f4-9baa-c43dc0a0fc34 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:54.074330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc2442b6-d94f-45f4-9baa-c43dc0a0fc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:54.130338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:54.176944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:54.463558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:54.465184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:54.612632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100ae8e-5cd2-484e-935e-1c616b63ed01 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:54.683478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:54.725182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:54.725708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:54.725988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:54.726258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:54.727866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:54.817067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:54.818679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5080-7d6d-4249-978f-05b81b02471c service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:54.837223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 708/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:55.726234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:55.726760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:55.726760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:55.728100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:55.733302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506b4c-23e7-4d8e-992c-545e95873420 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:55.733709 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506b4c-23e7-4d8e-992c-545e95873420 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:55.733857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506b4c-23e7-4d8e-992c-545e95873420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:55.809819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-21506b4c-23e7-4d8e-992c-545e95873420 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:55.811068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21506b4c-23e7-4d8e-992c-545e95873420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:55.839742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:55.910752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:56.060231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.061798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:56.206488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a306e-7ee0-4256-9bfd-39a69bb02c24 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.246776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:56.319792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.320309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.320606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.320879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.322457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:56.418796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.420411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab412a-3d0a-4387-9dbf-0b768ccb8412 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.439372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 710/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:57.332130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.332744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.332744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.334048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:57.338937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be3a5c-d868-48fb-9dff-9bc0ffc265dd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.339352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be3a5c-d868-48fb-9dff-9bc0ffc265dd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.339481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be3a5c-d868-48fb-9dff-9bc0ffc265dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.394471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:57.418196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c7be3a5c-d868-48fb-9dff-9bc0ffc265dd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:57.419380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c7be3a5c-d868-48fb-9dff-9bc0ffc265dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.519225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:57.676230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:57.678338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.837346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b312300-6ef1-4ded-b571-86057098b618 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:57.854353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:57.963930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:57.964446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.964781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.965197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.967128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:58.065846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:58.067459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb8ea2-565b-4345-be0e-473f826aed99 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:58.086310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 711/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:58.970381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.970946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.970946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.972291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:58.977297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9798-2989-4620-aff1-fb2eea6bccc1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.977701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9798-2989-4620-aff1-fb2eea6bccc1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.977880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9798-2989-4620-aff1-fb2eea6bccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.997388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:59.056173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8e8b9798-2989-4620-aff1-fb2eea6bccc1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:59.057423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8e8b9798-2989-4620-aff1-fb2eea6bccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.160383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:59.317276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.318935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.430037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:59.466585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a951f-b918-4ac1-8def-effb2b1d58cb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.577997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.578605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.578882 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.579883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.581433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.656726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:59.671526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.673710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aee521-63f1-4c1d-a6c0-307e47afda1e service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.692470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 713/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:00.565088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:00.565088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:00.565088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:00.566500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:00.571540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980c28-5d8b-40c2-9651-be7d8a97e3af None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:00.571957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980c28-5d8b-40c2-9651-be7d8a97e3af None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:00.572097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980c28-5d8b-40c2-9651-be7d8a97e3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:00.651987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-52980c28-5d8b-40c2-9651-be7d8a97e3af None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:00.653263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-52980c28-5d8b-40c2-9651-be7d8a97e3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:00.753980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:00.907186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:00.908732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:00.994714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:01.060886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8612b-74e2-4d47-a67d-afdf1841382d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:01.175116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:01.175719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:01.175997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:01.176271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:01.177900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:01.226771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:01.268604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:01.270244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270ab1e-f763-48b1-a643-5f6eb8bd3760 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:01.288208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 714/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:02.092314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.092916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.092916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.094407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:02.099786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975d13-f033-4aa5-9b7f-81c63e6df6c7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.100288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975d13-f033-4aa5-9b7f-81c63e6df6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.100401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc975d13-f033-4aa5-9b7f-81c63e6df6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.179199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dc975d13-f033-4aa5-9b7f-81c63e6df6c7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:02.180611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc975d13-f033-4aa5-9b7f-81c63e6df6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.279352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:02.432040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:02.433618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.483329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:02.586583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fff134-a328-46a0-a25e-fc3e59362bfa None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:02.706064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:02.706579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.707231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.707503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.709128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.716137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:02.807997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:02.809694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6286f92-ab3c-43bc-8da4-869d81a10dcd service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:02.831655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 716/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:03.662643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.663232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.663347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.664729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:03.669811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87531b87-f95a-496c-bd5d-c49644b03da4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.670211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87531b87-f95a-496c-bd5d-c49644b03da4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.670331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87531b87-f95a-496c-bd5d-c49644b03da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.749961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-87531b87-f95a-496c-bd5d-c49644b03da4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:03.751720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-87531b87-f95a-496c-bd5d-c49644b03da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:03.819549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:03.848789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:03.999418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.001445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:04.019414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:04.150654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe7593-2cb8-4e40-9c91-fe2b3c0e1ca7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.238899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:04.265446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:04.265983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.266265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.266574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.268316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:04.358010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.359613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f0fb7-39b1-42cc-80d9-b9a5e0e75ac7 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.378534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 717/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:05.207381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.207964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.207964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.209327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:05.214225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e872c6-b00e-456d-a040-4682cb00066b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.214667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e872c6-b00e-456d-a040-4682cb00066b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.214853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e872c6-b00e-456d-a040-4682cb00066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.291881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-62e872c6-b00e-456d-a040-4682cb00066b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:05.293131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-62e872c6-b00e-456d-a040-4682cb00066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.334579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:05.390100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:05.542544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:05.544070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.693450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a29105-91e1-43c0-881b-ff3300a25ea6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:05.749214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:05.808984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.809540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.809816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.810084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.811691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.901194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.902813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39cc92-7fbe-46c1-9491-6b65194463ff service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.921064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 719/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:06.726515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.727089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.727089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.728479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:06.734108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd0f8-972c-401e-8ace-f2994886211f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.734635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd0f8-972c-401e-8ace-f2994886211f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.734739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bd0f8-972c-401e-8ace-f2994886211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.825198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb7bd0f8-972c-401e-8ace-f2994886211f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:06.826468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb7bd0f8-972c-401e-8ace-f2994886211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:06.828766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:06.938006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:07.090843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.092612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.246445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90bf94-017a-42fd-9203-80e3d46ed926 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.275208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:07.367134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.367754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.368036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.368307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.369962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.464685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.466303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0c61-3e44-4a39-8dd6-0742a6547afa service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.484878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 720/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:09.333828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:09.334420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:09.334420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:09.335775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:09.341127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308864-5bb3-4818-b90e-261ef1008e1c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:09.341570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308864-5bb3-4818-b90e-261ef1008e1c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:09.341677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a308864-5bb3-4818-b90e-261ef1008e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:09.382532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:09.408788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9a308864-5bb3-4818-b90e-261ef1008e1c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:09.410000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a308864-5bb3-4818-b90e-261ef1008e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:09.506469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:09.793318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:09.794910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:09.903816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03032f78-341d-4ead-9ad5-142f316a4daf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:09.936204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:09.979732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:09.980339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:09.980647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:09.980926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:09.982537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:10.098555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:10.099928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5ab3e-9b34-4754-8ddc-a7197d1c684d service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:10.119537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 722/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:12.208925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.209538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.209538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.210851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:12.216146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ecca5-3116-41eb-84ef-55ab8cd0c1dc None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.216577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ecca5-3116-41eb-84ef-55ab8cd0c1dc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.216709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ecca5-3116-41eb-84ef-55ab8cd0c1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.282785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9d7ecca5-3116-41eb-84ef-55ab8cd0c1dc None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:12.283977 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d7ecca5-3116-41eb-84ef-55ab8cd0c1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:12.296479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:12.382676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:12.665192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:12.666779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:12.770584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8c398-709a-49d6-bdcd-e4f150fdd836 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:12.783552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:12.845891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:12.846500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.846776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.847044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.848686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:12.960023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:12.961409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2f60f-f4d3-49c8-93c2-38351703eae9 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:12.978936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 723/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:15.211459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:15.212011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:15.212011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:15.213335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:15.218940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a976-067d-4823-a13f-93d09e56a6a3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:15.219448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a976-067d-4823-a13f-93d09e56a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:15.219645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a976-067d-4823-a13f-93d09e56a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:15.392289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:15.412125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-88c4a976-067d-4823-a13f-93d09e56a6a3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:15.413397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-88c4a976-067d-4823-a13f-93d09e56a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:15.512249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:15.804571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:15.806153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:15.910431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b724c7-ab2a-4ca9-a978-e6bfedef4c88 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:15.949575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:15.984976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:15.985586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:15.985862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:15.986135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:15.987746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:16.096750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:16.098115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059061c-216d-4b9d-b583-90f0b3198c43 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:16.115959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 725/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:18.190403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.191086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.191086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.192208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:18.197947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108aaf-3eda-4ac3-89a3-fce07f530b12 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.198384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108aaf-3eda-4ac3-89a3-fce07f530b12 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.198505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7108aaf-3eda-4ac3-89a3-fce07f530b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.267842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d7108aaf-3eda-4ac3-89a3-fce07f530b12 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:18.269092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d7108aaf-3eda-4ac3-89a3-fce07f530b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:18.290565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:18.368226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:18.654128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.655667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:18.762158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2858017-3962-4dd3-bff5-3866f213bfc1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.784157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:18.838855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.839467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.839744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.840016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.841669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:18.954076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.955918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185441-ddde-4c77-8f4e-ca7b1053b69c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.974116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 726/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:20.999436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.000001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.000112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.001643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:21.007111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d70-2c41-4c3c-8a65-918a2244ebb3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.007544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d70-2c41-4c3c-8a65-918a2244ebb3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.007640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de262d70-2c41-4c3c-8a65-918a2244ebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.074874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:21.077726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-de262d70-2c41-4c3c-8a65-918a2244ebb3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:21.080007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-de262d70-2c41-4c3c-8a65-918a2244ebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:21.186894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:21.473894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:21.475461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:21.581961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4e5e6-64c1-4b07-8c67-5eec5ccd12bc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:21.630608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:21.656427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:21.657069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.657393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.657948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.659234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:21.771879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:21.773264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f42c-0532-48a7-8de5-d4f649ecc8c9 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:21.792306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 728/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:23.831566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:23.832109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:23.832109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:23.833540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:28:23.839497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006366-543e-43b9-beff-9a0b32310b6e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:23.840036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006366-543e-43b9-beff-9a0b32310b6e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:23.840036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae006366-543e-43b9-beff-9a0b32310b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:23.928786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ae006366-543e-43b9-beff-9a0b32310b6e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:23.930209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ae006366-543e-43b9-beff-9a0b32310b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:23.964415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:24.042118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:24.326919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:24.328462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.432697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40425e-249c-4bee-93ba-cd933cb36167 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:24.459944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:24.504759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:24.505369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:24.505649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:24.505947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:24.507570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.619308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:24.620708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1425ae-a4a7-45be-9763-709e928ebf14 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:24.638990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 729/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:26.607644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:26.608182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:26.608182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:26.609622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:26.615112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd05f-e56a-4637-b7cf-0b9f13c4a6e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:26.615577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd05f-e56a-4637-b7cf-0b9f13c4a6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:26.615676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dd05f-e56a-4637-b7cf-0b9f13c4a6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:26.686271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-926dd05f-e56a-4637-b7cf-0b9f13c4a6e9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:26.687522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-926dd05f-e56a-4637-b7cf-0b9f13c4a6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:26.697208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:26.788976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:27.077105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.078718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.199387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0262ed0-c15f-411e-8500-0797b1bd84b8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.257783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:27.277445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.278062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.278339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.278606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.280242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.407682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.410428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbee13b-f5bb-4a55-bfb1-2a271dc9c5f1 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.441998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 731/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:29.453633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.454123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.454240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.455564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:28:29.461847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e615fc-c091-43a1-8bde-bd3b51ed808f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.462309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e615fc-c091-43a1-8bde-bd3b51ed808f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.462393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e615fc-c091-43a1-8bde-bd3b51ed808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.529988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-95e615fc-c091-43a1-8bde-bd3b51ed808f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:29.531236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-95e615fc-c091-43a1-8bde-bd3b51ed808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.571089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:29.630418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:29.920056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.921627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.050874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb3fd3-a83a-493b-9f92-f556c84d9a1b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:30.091228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:30.130111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:30.130630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:30.130908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:30.131183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:30.132848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.246350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:30.247960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de410-2fe8-4049-a702-d75a9f60f868 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:30.268503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 732/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:32.257042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.257611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.257611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.258876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:32.264250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba2a5f-89df-420e-9742-ad12370d6c7e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.264740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba2a5f-89df-420e-9742-ad12370d6c7e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.264865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba2a5f-89df-420e-9742-ad12370d6c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.332008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-98ba2a5f-89df-420e-9742-ad12370d6c7e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:32.333400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-98ba2a5f-89df-420e-9742-ad12370d6c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.351994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:32.433447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:32.725509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.727105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.840001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a9c44-731f-4e12-a387-e32ee6bf99e4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.947826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:32.959790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.960834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.961313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.961785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.964766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:33.098752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.100194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117aa57-b751-4189-8367-473d90498aca service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.119611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 734/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:35.060136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.060753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.060753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.062179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:35.068177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3357c-86f2-49da-9157-1ec3f5713af4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.068639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3357c-86f2-49da-9157-1ec3f5713af4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.068860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3357c-86f2-49da-9157-1ec3f5713af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.135381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fbe3357c-86f2-49da-9157-1ec3f5713af4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:35.136622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fbe3357c-86f2-49da-9157-1ec3f5713af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.185416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:35.239395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:35.523660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:35.525481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.633757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e93b2-74ed-4d08-87fd-619bb46bf646 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:35.686278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:35.714452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:35.715067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.715414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.715626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.717323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.831582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:35.833094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79355127-2944-4c28-b220-586b2b0dcfef service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:35.851874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 735/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:45.237558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=22 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:45.849548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=66 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:55.995643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5766-226e-4ea3-9b99-e88347743612 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:55.996210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5766-226e-4ea3-9b99-e88347743612 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:55.996210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5766-226e-4ea3-9b99-e88347743612 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:55.997594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 736/1471] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:56.003465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a6c1-f1ee-409f-be79-7506fb0dace0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:56.003903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a6c1-f1ee-409f-be79-7506fb0dace0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:56.004009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274a6c1-f1ee-409f-be79-7506fb0dace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:56.073440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6274a6c1-f1ee-409f-be79-7506fb0dace0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:56.074659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6274a6c1-f1ee-409f-be79-7506fb0dace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:56.172789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 736/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:56.464050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:56.464331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:56.464636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:56.466005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 737/1473] 10.222.0.20 () {54 vars in 967 bytes} [Mon Aug 29 21:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:56.472255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4e6b1-3ea9-41cc-b1f2-66536fef019e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:56.472546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4e6b1-3ea9-41cc-b1f2-66536fef019e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:56.472852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4e6b1-3ea9-41cc-b1f2-66536fef019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:56.559764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5ca4e6b1-3ea9-41cc-b1f2-66536fef019e None None] MFA Rules not processed for user `0abda3580f52421d816eac7bee95b492`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:56.561015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5ca4e6b1-3ea9-41cc-b1f2-66536fef019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:56.582998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:56.642692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 737/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Mon Aug 29 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:56.652853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:56.655072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:56.786890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8652e-4925-4cef-a116-d6e8a2def294 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:56.873033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:56.873251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:56.873478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:56.873721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:56.875479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:56.938291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:57.004288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:57.005780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3cea8-8eeb-4bab-b598-a79be7f72590 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:57.026023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 738/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:59.046406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:59.046969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:59.046969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:59.048352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 738/1476] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:59.053839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635b71-2a3d-477c-a844-a783b8a2bb92 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:59.054292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635b71-2a3d-477c-a844-a783b8a2bb92 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:59.054396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c635b71-2a3d-477c-a844-a783b8a2bb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:59.120358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c635b71-2a3d-477c-a844-a783b8a2bb92 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:59.121484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c635b71-2a3d-477c-a844-a783b8a2bb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:59.216606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 739/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:59.484736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:59.486265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:59.616058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53620abc-3e39-4ec4-9a5a-71aad5787e98 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:59.665275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:59.711970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:59.712499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:59.712806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:59.713076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:59.713703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:59.715431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:28:59.715431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:28:59.715431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:28:59.716556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d1472-6c5d-421a-9bc4-c49f77725d9b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:59.735435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 739/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 29 21:28:59 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:29:01.713922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:01.714445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:01.714445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:01.715863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 740/1479] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:01.722147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d2b98-c951-44bf-9ce6-8d8e2a64dc6c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:01.722638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d2b98-c951-44bf-9ce6-8d8e2a64dc6c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:01.722638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d2b98-c951-44bf-9ce6-8d8e2a64dc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:01.799344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bd4d2b98-c951-44bf-9ce6-8d8e2a64dc6c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:01.800712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd4d2b98-c951-44bf-9ce6-8d8e2a64dc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:01.916688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 740/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:02.220327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:02.222006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:02.233482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:02.395277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ebd31-5f8d-4152-bcb4-e7595601b87d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:02.487204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:02.521747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:02.522099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:02.522375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:02.522645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:02.524523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:02.628764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:02.630396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47500d-eacb-485c-9758-60b85f66a7c6 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:02.654208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 741/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:04.790951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:04.791518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:04.791518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:04.792998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 741/1482] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:04.798321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3f664-54aa-43da-8b25-eabfe6668ec2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:04.798794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3f664-54aa-43da-8b25-eabfe6668ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:04.798904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3f664-54aa-43da-8b25-eabfe6668ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:04.867230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d9d3f664-54aa-43da-8b25-eabfe6668ec2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:04.868444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d9d3f664-54aa-43da-8b25-eabfe6668ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:04.916233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:04.969392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 742/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:05.261058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:05.262597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:05.286480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:05.367422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691f7b-e295-4142-9ab3-9582b73ab5ef None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:05.441290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:05.441898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:05.442175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:05.442444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:05.444250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:05.520170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:05.576175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:05.577635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48712b2c-d347-46f9-8849-29cb813c96e1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:05.596497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 742/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:07.581467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:07.582218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:07.582218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:07.584211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 743/1485] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:29:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:07.589763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962ad62-baae-4316-aadc-074e38a57ed6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:07.590246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962ad62-baae-4316-aadc-074e38a57ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:07.590340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962ad62-baae-4316-aadc-074e38a57ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:07.656931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9962ad62-baae-4316-aadc-074e38a57ed6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:07.658136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9962ad62-baae-4316-aadc-074e38a57ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:07.761790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 743/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:08.056018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:08.057604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:08.165342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8c139-6634-40b1-822b-4997e3a114dd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:08.240426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:08.242890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:08.243814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:08.244309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:08.244829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:08.246592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:08.360189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:08.361589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c4fe0-4e89-4edc-9d17-dc01291c9c99 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:08.379650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 744/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:10.461205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:10.461774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:10.461774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:10.463202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 744/1488] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:10.468990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4938c6-6424-4021-b8dc-32981398c87f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:10.469527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4938c6-6424-4021-b8dc-32981398c87f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:10.469527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4938c6-6424-4021-b8dc-32981398c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:10.540098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4a4938c6-6424-4021-b8dc-32981398c87f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:10.541372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4a4938c6-6424-4021-b8dc-32981398c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:10.596697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:10.643886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 745/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:10.936861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:10.938505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:10.980326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:11.053214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d95a7-2e2f-4468-b88d-e78a98ba5804 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:11.127777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:11.128296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:11.128569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:11.128860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:11.130451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:11.216953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:11.246856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:11.248288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f5d82-fa75-4629-a444-311af9002aec service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:11.266546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 745/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:13.403421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:13.403927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:13.403927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:13.405299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 746/1491] 10.222.0.20 () {54 vars in 927 bytes} [Mon Aug 29 21:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:13.410615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769837-6414-4b62-92d8-847583ecdb9c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:13.411100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769837-6414-4b62-92d8-847583ecdb9c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:13.411208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca769837-6414-4b62-92d8-847583ecdb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:13.478547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ca769837-6414-4b62-92d8-847583ecdb9c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:13.479778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ca769837-6414-4b62-92d8-847583ecdb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:13.581490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 746/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Mon Aug 29 21:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:13.878705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:13.880275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:13.988871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cfbd8-2392-451c-9106-8c37634d73d4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:14.070136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:14.070756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:14.071041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:14.071314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:14.072980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:14.082786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:14.193074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:14.194450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63554364-52fc-429c-aa86-af7607418d7c service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:14.213647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 747/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:23.579485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:24.211261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=50 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:27.428231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:27.428842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:27.428842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:27.505530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:27.506737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:27.603514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 747/1494] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:27.612113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:27.612650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:27.612750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:27.688890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:27.690015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:27.788393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 748/1495] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:27.792881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:27.794101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:27.805954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2056fb1-942a-4d73-ac9d-94e9a89d6977 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:27.878301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:27.915337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:27.915944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:27.916218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:27.916487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:27.917165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:30:27.917466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:27.918475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:27.918475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:30:27.918475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:30:27.919361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:27.930960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 748/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:30:27.939945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:27.954282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504bdaf-0bea-44b0-b611-ed5b53d4f226 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:28.141624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:28.175965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:28.176563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.176870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.177141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.177764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:28.179482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:28.179482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:30:28.179482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:30:28.180715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:28.241982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 749/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:30:27 2022] POST /identity/v3/projects => generated 379 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:30:28.246175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:28.247674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:28.282362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9e6b7-6b7f-49e7-ad37-6b25c8527bcd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:28.376320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:28.376961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.377268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.377514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.378150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:28.379154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:28.379154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:30:28.379154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:30:28.380356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:28.424516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 749/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:30:28 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:30:28.428938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:28.430471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:28.491525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ca00-f4c2-4287-aa46-5e98206c45ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:28.577116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:28.589344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:28.589951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.590230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.590503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.591145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:28.592166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:28.592166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:30:28.592166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:30:28.593152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:28.605835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 750/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:30:28.610556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:28.612140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:28.623556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38ce46-6dc6-47fe-b405-10118900e4eb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:28.649075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:28.720761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:28.721368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.721645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.721908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.722544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:28.723544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:28.723544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:30:28.723544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:30:28.724424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad1ee5-a112-43a3-8ed4-05968a88774e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:28.737342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 750/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:30:28.741608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:28.743171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:28.777890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b2bdb-2629-49e0-b9e5-c74801454d87 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:28.872983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:28.873840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.874124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.874401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] PATH_INFO: `/v3/projects/765a42974b8c42d4bdec0b18c28a5636/users/13f22f1d517f4cd1bc671e1c79ec3ec6/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.898378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=765a42974b8c42d4bdec0b18c28a5636, user_id=13f22f1d517f4cd1bc671e1c79ec3ec6, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:28.899721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:28.899721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:30:28.899721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:30:28.901683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca0631-8b97-4811-bc6f-f66e843d7e7c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:28.926786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 751/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:30:28 2022] PUT /identity/v3/projects/765a42974b8c42d4bdec0b18c28a5636/users/13f22f1d517f4cd1bc671e1c79ec3ec6/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:30:28.939350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcf2-0126-4387-b82f-05a567a1f632 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:28.939641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcf2-0126-4387-b82f-05a567a1f632 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:28.939919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0bcf2-0126-4387-b82f-05a567a1f632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:28.997298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-93b0bcf2-0126-4387-b82f-05a567a1f632 None None] MFA Rules not processed for user `13f22f1d517f4cd1bc671e1c79ec3ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:28.998589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-93b0bcf2-0126-4387-b82f-05a567a1f632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:29.079094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 751/1502] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:30:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:29.092287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:29.092615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:29.093149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:29.094806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 752/1503] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 29 21:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:30:29.099642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddedc6b-996a-4c69-b8c2-f36c19ddba0e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:29.099929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddedc6b-996a-4c69-b8c2-f36c19ddba0e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:29.100206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddedc6b-996a-4c69-b8c2-f36c19ddba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:29.131794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:29.177927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1ddedc6b-996a-4c69-b8c2-f36c19ddba0e None None] MFA Rules not processed for user `00f82e19c26b45d2a4ac373665f8c9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:29.179150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1ddedc6b-996a-4c69-b8c2-f36c19ddba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:29.255710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 752/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 29 21:30:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:29.263404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:29.266087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:29.320893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:29.377801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f62345-2a69-4c03-9306-d37414a27bcc None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:29.450725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:29.451250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:29.451528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:29.451799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:29.453976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:29.560874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:29.562370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:29.565992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:29.581966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 753/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:30:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:30:29.630999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424cc6-078c-45a7-82b8-be56346b9fd3 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:29.631512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424cc6-078c-45a7-82b8-be56346b9fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:29.631649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39424cc6-078c-45a7-82b8-be56346b9fd3 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:29.633000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 753/1506] 10.222.0.20 () {52 vars in 830 bytes} [Mon Aug 29 21:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:30:29.636799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:29.638047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:29.744571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c5e4d-9ddf-488d-87d8-22cadbdb7f07 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:29.844050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:29.844726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:29.844998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:29.845730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] PATH_INFO: `/v3/users/13f22f1d517f4cd1bc671e1c79ec3ec6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:29.861456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f22f1d517f4cd1bc671e1c79ec3ec6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:29.862231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:29.862231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:30:29.862231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:30:29.863496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:29.874124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:29.941390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f22f1d517f4cd1bc671e1c79ec3ec6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:29.965979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f22f1d517f4cd1bc671e1c79ec3ec6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:29.981494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f22f1d517f4cd1bc671e1c79ec3ec6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:29.982043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f22f1d517f4cd1bc671e1c79ec3ec6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:29.982601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f22f1d517f4cd1bc671e1c79ec3ec6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:29.997624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 754/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:30:29 2022] DELETE /identity/v3/users/13f22f1d517f4cd1bc671e1c79ec3ec6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:30:30.002896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:30.003182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:30.003457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:30.066550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:30.067790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:30.147371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:30.162199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 754/1508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:30.172389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:30.173701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:30.278802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d74a9b-12d0-4bfe-9ddf-5f28281e4c17 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:30.336308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:30.357403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:30.358019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:30.358299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:30.358567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:30.360713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:30.481707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:30.483153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210a2f7-5f50-4947-9684-c94da8ae3985 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:30.505341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 755/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:30:30.885811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:30.887031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:30.899470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb333b7-b33f-421e-a2cd-b84b3d0ce387 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:31.002644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:31.003268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:31.003543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:31.003809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] PATH_INFO: `/v3/projects/765a42974b8c42d4bdec0b18c28a5636` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:31.013706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765a42974b8c42d4bdec0b18c28a5636)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:31.014833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:30:31.014833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:30:31.014833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:30:31.016539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:31.075735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765a42974b8c42d4bdec0b18c28a5636', 'request_id': 'req-b5162b22-59ce-491d-a7a3-8358e7788653'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:31.082450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765a42974b8c42d4bdec0b18c28a5636', 'request_id': 'req-b5162b22-59ce-491d-a7a3-8358e7788653'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:31.093889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765a42974b8c42d4bdec0b18c28a5636', 'request_id': 'req-b5162b22-59ce-491d-a7a3-8358e7788653'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:31.123844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] The token cache is being invalidate because project 765a42974b8c42d4bdec0b18c28a5636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:30:31.124105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:31.124562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b5162b22-59ce-491d-a7a3-8358e7788653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:30:31.126036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 755/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:30:30 2022] DELETE /identity/v3/projects/765a42974b8c42d4bdec0b18c28a5636 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:30:40.502923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=62 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:41.122646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:02.800448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:02.801071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:02.801071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:02.802406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:02.807242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9bd46-e227-4fda-96e2-f2301e24f3b2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:02.807665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9bd46-e227-4fda-96e2-f2301e24f3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:02.807848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9bd46-e227-4fda-96e2-f2301e24f3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:02.890847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-01d9bd46-e227-4fda-96e2-f2301e24f3b2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:02.892025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-01d9bd46-e227-4fda-96e2-f2301e24f3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:03.004887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:03.159395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:03.161561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:03.290920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cce61a-7699-42e6-9d9d-bdc214da0a55 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:03.297534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:03.298152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:03.299074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:03.299358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:03.307695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:03.309484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ad809-9de4-4c18-895c-72d065642d73 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:03.318906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 757/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:04.179405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d878-8f8a-4025-afae-1488cbef3dd7 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:04.179970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d878-8f8a-4025-afae-1488cbef3dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:04.179970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235d878-8f8a-4025-afae-1488cbef3dd7 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:04.181350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:04.186788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c88e4-070c-4e45-afad-b29aee8e886d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:04.187255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c88e4-070c-4e45-afad-b29aee8e886d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:04.187410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c88e4-070c-4e45-afad-b29aee8e886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:04.207346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e09c88e4-070c-4e45-afad-b29aee8e886d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:04.208541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e09c88e4-070c-4e45-afad-b29aee8e886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:04.216722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:04.371645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235d878-8f8a-4025-afae-1488cbef3dd7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:04.387104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235d878-8f8a-4025-afae-1488cbef3dd7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:04.392969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:04.393568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:04.393850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:04.394132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:04.403273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:04.405042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdcac-cbec-45c2-ba27-9d1e3fee720d service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:04.415122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 758/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:32:05.151276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94483d0-933c-4b82-ac4e-e535f9d102b0 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:05.151507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94483d0-933c-4b82-ac4e-e535f9d102b0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:05.151734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94483d0-933c-4b82-ac4e-e535f9d102b0 None None] PATH_INFO: `/v3` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:05.153141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 759/1517] 10.222.0.72 () {54 vars in 925 bytes} [Mon Aug 29 21:32:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:32:05.159477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f29e-4f88-4f6a-a61c-4354f4b89b80 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:05.159757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f29e-4f88-4f6a-a61c-4354f4b89b80 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:05.160033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f29e-4f88-4f6a-a61c-4354f4b89b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:05.234522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d2b1f29e-4f88-4f6a-a61c-4354f4b89b80 None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:05.235745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2b1f29e-4f88-4f6a-a61c-4354f4b89b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:05.253903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:05.320003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 759/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Aug 29 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:09.169249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959b7e-d903-471f-944f-5d355f99a100 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:09.169814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959b7e-d903-471f-944f-5d355f99a100 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:09.169814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c959b7e-d903-471f-944f-5d355f99a100 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:09.171099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 760/1519] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 29 21:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:32:09.177925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406442-5e64-45b2-9e4a-518f64e31f07 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:09.178481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406442-5e64-45b2-9e4a-518f64e31f07 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:09.178481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406442-5e64-45b2-9e4a-518f64e31f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:09.198496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2d406442-5e64-45b2-9e4a-518f64e31f07 None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:09.199711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2d406442-5e64-45b2-9e4a-518f64e31f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:09.208469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 760/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 29 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:09.216332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c959b7e-d903-471f-944f-5d355f99a100 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:09.230320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c959b7e-d903-471f-944f-5d355f99a100 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:09.235399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:09.236576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:09.236904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:09.237170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:09.245694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:09.247283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9f91cf-d880-4e75-a190-3291b107f70b service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:09.257348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 761/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:15.658157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c264-3012-487b-826b-1eb4a0ac583f None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:15.658749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c264-3012-487b-826b-1eb4a0ac583f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:15.658749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c264-3012-487b-826b-1eb4a0ac583f None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:15.660088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:15.665321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e18c-97c5-4c4e-9fce-3e7019dccb98 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:15.665752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e18c-97c5-4c4e-9fce-3e7019dccb98 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:15.665877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4e18c-97c5-4c4e-9fce-3e7019dccb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:15.685475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a6d4e18c-97c5-4c4e-9fce-3e7019dccb98 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:15.686700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a6d4e18c-97c5-4c4e-9fce-3e7019dccb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:15.695186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:15.853423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c264-3012-487b-826b-1eb4a0ac583f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:15.873017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c264-3012-487b-826b-1eb4a0ac583f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:15.879317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:15.879928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:15.880211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:15.880487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:15.889556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:15.891166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0cff9-64e6-42e8-a260-885bd18ef7c8 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:15.902036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 762/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:16.726321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33d9c-4831-4c20-b9f7-fddfb7c7b0c1 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:16.726893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33d9c-4831-4c20-b9f7-fddfb7c7b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:16.726893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b33d9c-4831-4c20-b9f7-fddfb7c7b0c1 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:16.728217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:16.733079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46c12-e238-46aa-9c9e-b599b735b7d3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:16.733553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46c12-e238-46aa-9c9e-b599b735b7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:16.733654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e46c12-e238-46aa-9c9e-b599b735b7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:16.761205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-35e46c12-e238-46aa-9c9e-b599b735b7d3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:16.762470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-35e46c12-e238-46aa-9c9e-b599b735b7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:16.770549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:16.955968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b33d9c-4831-4c20-b9f7-fddfb7c7b0c1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:16.971478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b33d9c-4831-4c20-b9f7-fddfb7c7b0c1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:16.977186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:16.977729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:16.978009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:16.978284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:16.987189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:16.988864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf908035-3d27-4dfa-ab47-6c3108e1138d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:17.000351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 764/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:21.037644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:21.038240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:21.038240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:21.039549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:32:21.044740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30019bd6-961d-4bec-9085-1e8d0a3b2f8d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:21.045154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30019bd6-961d-4bec-9085-1e8d0a3b2f8d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:21.045289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30019bd6-961d-4bec-9085-1e8d0a3b2f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:21.065705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-30019bd6-961d-4bec-9085-1e8d0a3b2f8d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:21.066931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-30019bd6-961d-4bec-9085-1e8d0a3b2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:21.074913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:21.362425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:21.364132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:21.441827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacbf85-bd0c-4e16-b124-719a1ab8e355 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:21.445455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:21.446060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:21.446342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:21.446614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:21.455974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:21.457418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef9054-c13f-4f56-8fa4-59be1ced96fe service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:21.466742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 765/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:32:23.408169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3207bf2-d27c-41a5-8cbf-672ece07b4f2 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:23.409295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3207bf2-d27c-41a5-8cbf-672ece07b4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:23.409295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3207bf2-d27c-41a5-8cbf-672ece07b4f2 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:23.410812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:23.417285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855188-35a0-4598-8588-64de23dd6689 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:23.417816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855188-35a0-4598-8588-64de23dd6689 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:23.417816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7855188-35a0-4598-8588-64de23dd6689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:23.441946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b7855188-35a0-4598-8588-64de23dd6689 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:23.443101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b7855188-35a0-4598-8588-64de23dd6689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:23.451377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:23.737891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3207bf2-d27c-41a5-8cbf-672ece07b4f2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:23.748557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3207bf2-d27c-41a5-8cbf-672ece07b4f2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:23.752691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:23.753314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:23.753599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:23.753875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:23.763303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:23.764766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2be89-fee9-47be-9779-f370583c4947 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:23.774415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 767/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:27.265250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb25df-8b91-4213-9597-36dca779110d None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:27.265769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb25df-8b91-4213-9597-36dca779110d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:27.265769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb25df-8b91-4213-9597-36dca779110d None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:27.267147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:27.273141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219894a-9cac-4780-92ad-bc8e6ca1a122 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:27.273566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219894a-9cac-4780-92ad-bc8e6ca1a122 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:27.273687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219894a-9cac-4780-92ad-bc8e6ca1a122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:27.293461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b219894a-9cac-4780-92ad-bc8e6ca1a122 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:27.294671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b219894a-9cac-4780-92ad-bc8e6ca1a122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:27.303052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:27.860091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb25df-8b91-4213-9597-36dca779110d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:27.862443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13bb25df-8b91-4213-9597-36dca779110d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:27.927337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:27.940430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb25df-8b91-4213-9597-36dca779110d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:27.943965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:27.944563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:27.944999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:27.945273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:27.954560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:27.956066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458fd72-be99-4482-a642-050886d2658e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:27.965470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 768/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:29.915357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579347-111f-476f-89d1-cf726c11298a None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:29.915899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579347-111f-476f-89d1-cf726c11298a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:29.915899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a579347-111f-476f-89d1-cf726c11298a None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:29.917353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:32:29.922641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efcba7-6f8f-4435-9a50-e11d13290dfb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:29.923056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efcba7-6f8f-4435-9a50-e11d13290dfb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:29.923201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efcba7-6f8f-4435-9a50-e11d13290dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:29.943157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90efcba7-6f8f-4435-9a50-e11d13290dfb None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:29.944361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90efcba7-6f8f-4435-9a50-e11d13290dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:29.952312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:30.509807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a579347-111f-476f-89d1-cf726c11298a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:30.511778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0a579347-111f-476f-89d1-cf726c11298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:30.519565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:30.526203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a579347-111f-476f-89d1-cf726c11298a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:30.529720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:30.530314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:30.530595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:30.530863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:30.539965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:30.541407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf8d94-3f16-4ee0-9f88-e383b19624c0 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:30.551378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 770/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:39.938247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=11 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:40.546755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:57.300325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c4c82-5ad1-4977-95b6-a48de82f7cbd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.301004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c4c82-5ad1-4977-95b6-a48de82f7cbd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.301004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c4c82-5ad1-4977-95b6-a48de82f7cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.302231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcbac-0223-4eac-b215-c2aefb7bce14 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.303059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcbac-0223-4eac-b215-c2aefb7bce14 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.303563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafcbac-0223-4eac-b215-c2aefb7bce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.329797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1fafcbac-0223-4eac-b215-c2aefb7bce14 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.331488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1fafcbac-0223-4eac-b215-c2aefb7bce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.335330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-775c4c82-5ad1-4977-95b6-a48de82f7cbd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.336957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-775c4c82-5ad1-4977-95b6-a48de82f7cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.347000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 771/1540] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:57.350733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 770/1541] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.356237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45215de1-e069-4ba9-bf80-3142ae74d56c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.356856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45215de1-e069-4ba9-bf80-3142ae74d56c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.357359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45215de1-e069-4ba9-bf80-3142ae74d56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.359789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47bc0-5da6-4b92-a85c-e30a2de6ea9f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.360075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47bc0-5da6-4b92-a85c-e30a2de6ea9f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.360354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47bc0-5da6-4b92-a85c-e30a2de6ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.386975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-45215de1-e069-4ba9-bf80-3142ae74d56c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.388569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-45215de1-e069-4ba9-bf80-3142ae74d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.393608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f4b47bc0-5da6-4b92-a85c-e30a2de6ea9f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.394817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f4b47bc0-5da6-4b92-a85c-e30a2de6ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.404598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 772/1542] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.405879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.406167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.406444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.413196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 771/1543] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.414739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a115ce-4f43-4bff-a74e-8f4a83219cd7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.415399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a115ce-4f43-4bff-a74e-8f4a83219cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.416007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a115ce-4f43-4bff-a74e-8f4a83219cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.449695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.450961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.452077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c7a115ce-4f43-4bff-a74e-8f4a83219cd7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.453662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7a115ce-4f43-4bff-a74e-8f4a83219cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.461302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 773/1544] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.462246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.463723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 772/1545] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.465213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd2042-6027-4380-b359-6964677cf497 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.465737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd2042-6027-4380-b359-6964677cf497 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.466333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd2042-6027-4380-b359-6964677cf497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.475227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c133ad-c745-49a9-907e-bb7570c59027 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.481117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.481905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.482188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.482460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.483108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.483408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.484430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:33:57.484430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:33:57.484430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:33:57.485415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.489024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c9cd2042-6027-4380-b359-6964677cf497 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.490263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c9cd2042-6027-4380-b359-6964677cf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.497634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 774/1546] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.498436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.498940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 773/1547] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.499913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5318e-581a-408e-b8bf-a17dd4dddb17 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.500196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5318e-581a-408e-b8bf-a17dd4dddb17 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.500472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5318e-581a-408e-b8bf-a17dd4dddb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.511133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6fefc-d96f-491d-aaaa-6e7ddc6165ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.515744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.516384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.516725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.516970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.517612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.517913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.519486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a187c6-9cfa-462f-864a-09d148fbfa62 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.530411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e4e5318e-581a-408e-b8bf-a17dd4dddb17 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.531617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4e5318e-581a-408e-b8bf-a17dd4dddb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.531895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 775/1548] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.539246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f967-f8b6-4c31-9d23-588de6b84ea9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.539246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f967-f8b6-4c31-9d23-588de6b84ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.539246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8f967-f8b6-4c31-9d23-588de6b84ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.545623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 774/1549] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.546846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e436-da70-441d-9380-3b8fe770da2e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.547256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e436-da70-441d-9380-3b8fe770da2e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.547540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8e436-da70-441d-9380-3b8fe770da2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.558109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-92c8f967-f8b6-4c31-9d23-588de6b84ea9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.559304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-92c8f967-f8b6-4c31-9d23-588de6b84ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.568020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 776/1550] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.569286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.569569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.569840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.576250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-dad8e436-da70-441d-9380-3b8fe770da2e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.577540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dad8e436-da70-441d-9380-3b8fe770da2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.586672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 775/1551] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.587976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0223-a4b9-49a3-963b-6cadc8cd0ed6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.588267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0223-a4b9-49a3-963b-6cadc8cd0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.588540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de0223-a4b9-49a3-963b-6cadc8cd0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.590257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.591455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.599541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 777/1552] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.600287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.609307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-85de0223-a4b9-49a3-963b-6cadc8cd0ed6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.609755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c606bbf-a441-41c8-a70d-4e15b0a417a0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.610481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-85de0223-a4b9-49a3-963b-6cadc8cd0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.614962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.615552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.615828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.616098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.616753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.618854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3e30-d57b-4953-abbe-3e3ec2662422 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.619024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 776/1553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.620182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.620464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.620772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.641969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.643159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.651598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 777/1554] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.652303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.661938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2c75d-eb30-4299-8012-eb5c05ad5dc6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.663150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 778/1555] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:57.664383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40207c2f-9d79-43c0-b9ec-e6cc7bbf4d01 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.664562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40207c2f-9d79-43c0-b9ec-e6cc7bbf4d01 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.664859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40207c2f-9d79-43c0-b9ec-e6cc7bbf4d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.685071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-40207c2f-9d79-43c0-b9ec-e6cc7bbf4d01 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.686272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40207c2f-9d79-43c0-b9ec-e6cc7bbf4d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.722099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.722633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.723025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.723301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.723962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.725074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:33:57.725074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:33:57.725074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:33:57.726335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.751559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 779/1556] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.757110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56cd8a-3542-4de1-9a5d-4b444e239c15 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.757401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56cd8a-3542-4de1-9a5d-4b444e239c15 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.757679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56cd8a-3542-4de1-9a5d-4b444e239c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.761518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 778/1557] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:57.761518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.780648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9d56cd8a-3542-4de1-9a5d-4b444e239c15 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.781880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9d56cd8a-3542-4de1-9a5d-4b444e239c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.827875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68384-69c3-4ccc-b917-bb2ae9a14409 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.832545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.833316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.833587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.833859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.834505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.834809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.836420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.856089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 779/1558] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.856089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.856377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 780/1559] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.856377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.856377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.856377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.865080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3af759-14a2-48fb-9cf0-3895c77ffad9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.870895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.871513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.871794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.872067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.872747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.873121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.874176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:57.874932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.875528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.884744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 781/1560] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:57.886017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.893345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 780/1561] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.894853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.895202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:57.900688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d792e-b5fc-4f9f-8889-ff07b595b6ac None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.908049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.908783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.909068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.909362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.910011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.910138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a095284-784a-4fff-9597-867fded29304 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.910315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.912468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.915406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.916018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.916294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.916572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.917258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.919178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.925356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 782/1562] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.926584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.936963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:57.938763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273841f-ff7d-47e1-8d78-37a148262391 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.945653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.946754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.947225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.947688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.948751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.949158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.952829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 781/1563] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:57.952829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.953065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.960795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354ec2e-522d-45cf-96a2-f4a90da0614c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.965574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.966179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.966458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.966725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.967193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 783/1564] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:57.967346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.968027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.969395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.979909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12dbb5-d1f1-4a88-9d86-e3744d86032d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.985062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.985692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.986008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.986467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.987247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:57.987642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.989847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.996152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 782/1565] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:57.996760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.003116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 784/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:58.003834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.011239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de9f7-b562-4022-b4dd-bd8c3878bda0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.016044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.016689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.016859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5334c-b1f0-4f31-9665-4b3076e0f521 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.016990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.017240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.017881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.019806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.022475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.023081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.023363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.024146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.024146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:58.024824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.026479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.054376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 785/1567] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:58.054376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.054691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 783/1568] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.054691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.062975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4240-ec2a-44f0-be0e-1480cb9978ee admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.125640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6b9d5-4a87-4bf7-9b62-9c3bd2c5a4c0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.126378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.127018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.127304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.127720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.128356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.130460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.130839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.131438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.131717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.131987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.132649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.134533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.160217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 786/1569] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.161578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.165624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 784/1570] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.165624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.225887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224112f-a2ff-4029-945d-30b4ccbe7e95 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.231191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.231767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.232061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.232338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.232697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7449fd-9345-41f5-b41a-59dd18618bc2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.233061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.234766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.237915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.238610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.238939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.239832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.240759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.242916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.249307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 787/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.249307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.257754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207628ea-d695-4f06-a022-25605c6ac912 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.263518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.264116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.264398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.264808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.265519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.267288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.272703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 785/1572] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.277225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.279077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 788/1573] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.279732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.289754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771104-dbf0-4e80-a260-ea49cf0364dd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.308982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.312148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.349555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecb40b-af62-4fe4-857b-8625ae8be12a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.352996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.353602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.353872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.354139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.354404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.354767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.355219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.355633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.356002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.356697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.356887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.358656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.387677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 789/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.388838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.396766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 786/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.396766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.453797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260c14-c680-46ae-94bd-0951bd5aba74 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.459157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.459773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.460051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.460324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.460437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33c6ea-687d-4f49-bcef-569ab30976f7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.461084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.462339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:33:58.462339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:33:58.462339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:33:58.463780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.467159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.467794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.468077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.468354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.469072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.471043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.487134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 790/1576] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.487904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.498920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44532b-8e47-42dc-a462-b33cc0a4717e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.500612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 787/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.501525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.508495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.509558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.510069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.510540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.511630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.513074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ceaaba-f591-4e83-92af-4651d79225a0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.515221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.517805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.518404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.518685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.518960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.519601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.521971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.534733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 788/1578] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.535390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.546495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69dce9-7ac4-4a14-aaec-21e42edcd3d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.548790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 791/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 288 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.549695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.551138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.551788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.552093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.552340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.553018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.554979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.563645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f1795-a95b-491f-9835-c6fb07e05699 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.569053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.569914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.570200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.570473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.571272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.573029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.580972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 789/1580] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 282 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.580972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.587781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 792/1581] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.590742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f321e-fc7a-4e54-aa3b-fb1649824528 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.591633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.595417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.596470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.596470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.596765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.597230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.599724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.603117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3aaf1-808e-494a-9496-71b2c7b84775 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.607821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.608439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.608755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.609031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.609669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.611595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.623802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 790/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.624902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.635860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0be70-165b-45bc-92ec-1d2f7f7e390d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.641990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.645271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 793/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.646217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.646217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.646217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.646462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.646553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.646553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.660420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 791/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.660420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.660751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccc6d8-053f-415e-bf36-d6267f534016 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.666581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.667196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.667476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.667749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.668395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.669475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66267796-ca14-4b9f-b67f-79ca5d2b4ff5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.670050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.674377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.675237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.675354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.675615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] PATH_INFO: `/v3/projects/cb5424a2fd9d45c899b4ff8b7869f50e/users/b8ac60b941b04b2eb01281ba5f0e0b96/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.680752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 794/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.681603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.695276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b7c45-5656-40ad-9796-46b550ee1975 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.696823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5424a2fd9d45c899b4ff8b7869f50e, user_id=b8ac60b941b04b2eb01281ba5f0e0b96, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.697898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:33:58.697898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:33:58.697898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:33:58.699842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.701858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.702591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.702867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.703143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.703792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.705530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.715757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 792/1586] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:58 2022] PUT /identity/v3/projects/cb5424a2fd9d45c899b4ff8b7869f50e/users/b8ac60b941b04b2eb01281ba5f0e0b96/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:58.717187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.722187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 795/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.722187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.730049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb51bb-606c-4fa9-91f2-38a42982933d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.745579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.758560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.793070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.793472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36297-f9e1-4202-b7d8-536c02429c7c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.793603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.793877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.794339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] PATH_INFO: `/v3/projects/57c7abc43eb142d3b36a6d91beee5d10/users/5344cf7b391d44beac2c602ac3e2e64b/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.798515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.799150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.799463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.799705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.800357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.802043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.809962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c7abc43eb142d3b36a6d91beee5d10, user_id=5344cf7b391d44beac2c602ac3e2e64b, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.813722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 793/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.813907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.814006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.826035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8c3e-7db5-404a-9ec0-060d96c7f6d3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.830344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 796/1589] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:58 2022] PUT /identity/v3/projects/57c7abc43eb142d3b36a6d91beee5d10/users/5344cf7b391d44beac2c602ac3e2e64b/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:58.831316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.890560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.891605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.891891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.892164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.892828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.894440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.898154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305139-d645-4852-aef1-6b08fa76ad2b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.903230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.903843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.904125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.904400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.905092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.905510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 794/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.906322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.906783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.916392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cac609-c6df-4fc6-a2aa-151a05602822 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.918083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 797/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.918083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.921773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.922376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.922698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.922992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.923630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.925263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2e79-fb81-4b88-8e2e-479caa4c7e67 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.929029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687be10b-acb3-43a2-8fab-b80e3f24dd1a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.934269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.934855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.935133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.935407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.938699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 795/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.938823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.938923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.938923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.938923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.939236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.951408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 798/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.951408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.957406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:58.959179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.960751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def65419-90ce-4791-bca0-644970fc816f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.965580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.966162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.966441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.966712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.967333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.967427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 796/1594] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:58.968211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.968963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff3705-d68e-4228-907f-4f4c6bd66cc9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.980624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 799/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.980821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d35cd-cb48-405a-895e-bdac0e86bb94 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.981207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.981628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.981914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.985494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.986102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.986411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.986685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.987323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.988950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.000017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 797/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:59.000607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.008940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.010230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.010744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47d045-e18b-4590-927f-707d86a2394c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.015303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.015892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.016177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.016447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.017113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.018336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 800/1597] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.018694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.019187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.029974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a9580-72ed-4286-9dba-14a4c254ef2a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.033926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 798/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:59.034940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.035083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.035549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.035831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.036176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.036904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.038503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.046069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ae107-de73-4e88-a910-4957295e946a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.049424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 801/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:59.050273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.050999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.053929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.054237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.054518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] PATH_INFO: `/v3/projects/e3fc6dcea8f34aad91a35e71fb44daac/users/4ed98b05cbf64b55ad8d2443ae1aa344/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.061955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393e60d-6e25-4cb4-95db-fb62bdf7332c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.066688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.068065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.068529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.068846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] PATH_INFO: `/v3/projects/5220f1dfa6a74754bd08f925c85847b2/users/89b272427c6444b49293e5046c686ffc/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.082407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fc6dcea8f34aad91a35e71fb44daac, user_id=4ed98b05cbf64b55ad8d2443ae1aa344, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.083858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5220f1dfa6a74754bd08f925c85847b2, user_id=89b272427c6444b49293e5046c686ffc, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.085252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.086886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.099579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 799/1600] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/e3fc6dcea8f34aad91a35e71fb44daac/users/4ed98b05cbf64b55ad8d2443ae1aa344/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.100611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.101366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 802/1601] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/5220f1dfa6a74754bd08f925c85847b2/users/89b272427c6444b49293e5046c686ffc/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.103172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.154606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:59.156409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:59.160653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a5639-34ff-4b46-92a0-09ceba06b02b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.173815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ccd9-82a7-4fdb-a7c7-31bed87afb7b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.178975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.179619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.180047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.180259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] PATH_INFO: `/v3/projects/7d9928c37b2b46f5abd0ac1ba1c3ca92/users/6f0d97b196f44ec985db30f86d4ad3f4/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.195689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9928c37b2b46f5abd0ac1ba1c3ca92, user_id=6f0d97b196f44ec985db30f86d4ad3f4, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.198365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.211533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 803/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/7d9928c37b2b46f5abd0ac1ba1c3ca92/users/6f0d97b196f44ec985db30f86d4ad3f4/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.212400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.215958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.216571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.217022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.217271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.227775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.229351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.239865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 800/1603] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:59.240742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.248943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f733a8b-4df0-49af-b653-7a3744194751 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.282620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0adbb-170f-40df-8d92-1217d1d01946 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.291245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.291916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.292192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.292466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] PATH_INFO: `/v3/projects/45d62a4c1ecb4a8bbf8c71ca5ba6dbb6/users/c26b72a7bb864d188dc600e8edd52ed5/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.308166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d62a4c1ecb4a8bbf8c71ca5ba6dbb6, user_id=c26b72a7bb864d188dc600e8edd52ed5, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.309120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.309869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.310147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.310416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.311238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.321767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.323606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.338851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 804/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/45d62a4c1ecb4a8bbf8c71ca5ba6dbb6/users/c26b72a7bb864d188dc600e8edd52ed5/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.338851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.339168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 801/1605] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:59.339168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.344768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c33d-2286-4499-a1e5-7c3c161405ed service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.409719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.410538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.411481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.412102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] PATH_INFO: `/v3/projects/23e5e8ff1d1f45a1a9d1f01889f496b1/users/6f1bdcc244f9484f8aca8f5e59839027/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.416275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e32cc-7ac6-4bd9-99c7-49586f6ea1c4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.421560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.422679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.422955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.423225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] PATH_INFO: `/v3/projects/09369e28336d4f49a60ce03c3a3009ac/users/55a5c6359109440fbb66d985fafc24bf/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.431663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e5e8ff1d1f45a1a9d1f01889f496b1, user_id=6f1bdcc244f9484f8aca8f5e59839027, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.434785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f172dc-3964-460c-95a8-0516e2291ac5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.442891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09369e28336d4f49a60ce03c3a3009ac, user_id=55a5c6359109440fbb66d985fafc24bf, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.445575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f8cb5-3dcc-4711-9e56-d8af618dd186 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.449852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 802/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/23e5e8ff1d1f45a1a9d1f01889f496b1/users/6f1bdcc244f9484f8aca8f5e59839027/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.451196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c154f-063f-446f-8a57-a228be0e0ac1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.451651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c154f-063f-446f-8a57-a228be0e0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.451763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c154f-063f-446f-8a57-a228be0e0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.470066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 805/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:59 2022] PUT /identity/v3/projects/09369e28336d4f49a60ce03c3a3009ac/users/55a5c6359109440fbb66d985fafc24bf/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:59.471469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861313-103c-432e-af44-4f32764cc646 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.472215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861313-103c-432e-af44-4f32764cc646 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.472499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861313-103c-432e-af44-4f32764cc646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.479525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-019c154f-063f-446f-8a57-a228be0e0ac1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.481068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-019c154f-063f-446f-8a57-a228be0e0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.504176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-26861313-103c-432e-af44-4f32764cc646 None None] MFA Rules not processed for user `4ed98b05cbf64b55ad8d2443ae1aa344`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.505553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-26861313-103c-432e-af44-4f32764cc646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.516463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:59.530473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:59.561061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 803/1608] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.562159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76d72-98b3-4a97-b0fc-50008c4c4ac1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.562443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76d72-98b3-4a97-b0fc-50008c4c4ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.562714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd76d72-98b3-4a97-b0fc-50008c4c4ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.585607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 806/1609] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.590823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfc16a-c354-46a5-850a-fca9d2a82a67 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.591304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfc16a-c354-46a5-850a-fca9d2a82a67 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.591575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6bd76d72-98b3-4a97-b0fc-50008c4c4ac1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.591985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfc16a-c354-46a5-850a-fca9d2a82a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.593005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6bd76d72-98b3-4a97-b0fc-50008c4c4ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.602090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 804/1610] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.603361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.603714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.604081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.635332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bccfc16a-c354-46a5-850a-fca9d2a82a67 None None] MFA Rules not processed for user `c26b72a7bb864d188dc600e8edd52ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.637363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bccfc16a-c354-46a5-850a-fca9d2a82a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.640645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] MFA Rules not processed for user `6f1bdcc244f9484f8aca8f5e59839027`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.641940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.729278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 807/1611] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.730837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.731188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.731523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.748419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 805/1612] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.750883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.752729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.783379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:59.786971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.812102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 808/1613] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:59.812102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.833494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db03e65-e92b-4e00-b1cb-14d50513674c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.839448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.840243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.840651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.841025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.841775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.843816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.848434 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f8d6-f7e4-49ef-bb65-064972d09cf1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.853891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.854753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.855210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.855704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.872763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.874836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.887919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 806/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:59.888908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.892879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 809/1615] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:59.894018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:59.912000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb533b-74b8-4c59-ad01-40452f506cd4 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.999463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.000439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.001148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.001438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.018576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a026d43-3e3b-4308-93b0-4e028d6d1930 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.025282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.025972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.026444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.026726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.027436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.030396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a256d-7473-4d0c-8f40-fe48429566f6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.042686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:00.070020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 810/1616] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:59 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:00.070020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5a7cc-aeba-47cc-81f9-cc142ccc94b9 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.070020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5a7cc-aeba-47cc-81f9-cc142ccc94b9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.070020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5a7cc-aeba-47cc-81f9-cc142ccc94b9 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.071949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 811/1617] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 29 21:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:00.073639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5a7cc-aeba-47cc-81f9-cc142ccc94b9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.088349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:00.131914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.133855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.149841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 807/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:00.150664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.164520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d00eb-44b6-475e-86e8-7d44165d6d3e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.167058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5a7cc-aeba-47cc-81f9-cc142ccc94b9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.172400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.173113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.173683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.173968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.234482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.235148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.235503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.235843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.236542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.238582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.261911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.263769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd46e14-7459-4f60-8845-c30a6e6730a0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.265059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 808/1619] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:00 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:00.265889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.282827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f40dac-d1f6-4023-96f0-711b2d98a88a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.283674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 812/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:00.285185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.285627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.286146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.287785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.288517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.289081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.289423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.290128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.292178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.317777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 809/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:00 2022] POST /identity/v3/users => generated 288 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:00.320103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.331649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e983d5-8c19-4df0-ba56-bd7950e2af5e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.338656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:00.339906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:00.340530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.341792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.342219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.342513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.343540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.344885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.356483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 810/1622] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:00.360563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.372575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ce30e-265d-4a15-95ff-79e6fbcaa359 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.379177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.381157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.381542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.382368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.382675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.384429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.395598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 811/1623] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:00.397015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.409616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbecaf-7925-4cf6-8645-460bf6f73dd4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.414736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.415750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.415972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.416347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] PATH_INFO: `/v3/projects/6a5291ee2b604a089ae9ecf52210d556/users/d2a3c5dc8a8642429195ce710839c569/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.435490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5291ee2b604a089ae9ecf52210d556, user_id=d2a3c5dc8a8642429195ce710839c569, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.436039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 813/1624] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 29 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:00.436988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.439174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.460892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f7dd2-ac6c-4347-b5bf-d418d12d9c72 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.467166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 812/1625] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:00 2022] PUT /identity/v3/projects/6a5291ee2b604a089ae9ecf52210d556/users/d2a3c5dc8a8642429195ce710839c569/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:00.468479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.523758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:00.528083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:00.561200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.561795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.562082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.562364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] PATH_INFO: `/v3/projects/d5e4357d29684686b438d150fcbb5ccf/users/a63f6d28100349598e805d738264e4df/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.576630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81dd46-2530-4fdc-9434-9ad92fa1ddec admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.588676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.588973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.589272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.590027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.591395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e4357d29684686b438d150fcbb5ccf, user_id=a63f6d28100349598e805d738264e4df, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.596470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.617323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 814/1626] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:00 2022] PUT /identity/v3/projects/d5e4357d29684686b438d150fcbb5ccf/users/a63f6d28100349598e805d738264e4df/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:00.621684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.683185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.685601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.690876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7374571-9d46-4724-9985-8c5db3f3ebcc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.698028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.698958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.699690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.700548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.701166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.703286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.704970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 813/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:00.707294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.721250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 815/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:00.722571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.729211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad6965-c241-44ad-b798-13d2c77ef6c5 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.735725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1a553-dac7-4365-8f22-40b5be8ac71f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.741482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.742284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.742796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.743213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.743441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.743693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.744045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.744174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] PATH_INFO: `/v3/projects/6a5291ee2b604a089ae9ecf52210d556/users/d2a3c5dc8a8642429195ce710839c569/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.745185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.747171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5291ee2b604a089ae9ecf52210d556, user_id=d2a3c5dc8a8642429195ce710839c569, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.747730 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.752772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391adb9-c6ea-4237-8c40-3c206eae869b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.769257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 814/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:00.771271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.781781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 816/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:00 2022] PUT /identity/v3/projects/6a5291ee2b604a089ae9ecf52210d556/users/d2a3c5dc8a8642429195ce710839c569/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:00.785706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.786251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.788093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.797328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae6b43-6594-471d-917c-e30eeb56c380 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:00.845647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] MFA Rules not processed for user `d2a3c5dc8a8642429195ce710839c569`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:00.847225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:00.902158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:00.903559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.904106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.904519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] PATH_INFO: `/v3/projects/d5e4357d29684686b438d150fcbb5ccf/users/a63f6d28100349598e805d738264e4df/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.908930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e4357d29684686b438d150fcbb5ccf, user_id=a63f6d28100349598e805d738264e4df, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:00.914321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5c7e-241b-4de3-afbc-0fba9638b0f2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:00.934490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 815/1631] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:00 2022] PUT /identity/v3/projects/d5e4357d29684686b438d150fcbb5ccf/users/a63f6d28100349598e805d738264e4df/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:00.946144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b053362-4a39-42c3-a44a-9f7394e2647e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:00.946537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b053362-4a39-42c3-a44a-9f7394e2647e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:00.946820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b053362-4a39-42c3-a44a-9f7394e2647e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:00.977912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 817/1632] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:00.993433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:00.997985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3b053362-4a39-42c3-a44a-9f7394e2647e None None] MFA Rules not processed for user `a63f6d28100349598e805d738264e4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:00.999592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3b053362-4a39-42c3-a44a-9f7394e2647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:01.042766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:01.079094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8037e-50f4-4174-b41e-d0a14de75681 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.081021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:01.127702 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 816/1633] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:01.130790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ad12-cc23-4835-a0de-0080d413ed0a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.132550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ad12-cc23-4835-a0de-0080d413ed0a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.133089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ad12-cc23-4835-a0de-0080d413ed0a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.134699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 817/1634] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 29 21:34:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:01.143025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.143359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.143632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.201143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:01.202384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.203290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:01.204223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.206135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.206538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.371279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.372909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 818/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:01.374512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.380891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.393071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeecb55-951c-4186-86bb-ad29e36e8a95 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.398496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 818/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:01.406383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.407242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.407669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.408088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.425948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.427592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.462133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 819/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:01.508177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.521566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2894d-8c7c-46d3-8b9e-e2ea7ccaa3cc service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.535548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.537128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.537435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.537706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.581476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.588109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.625018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 819/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:02.751383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:02.865692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:02.920239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1e8-de03-4336-a9bb-b13da59dbd7d service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:02.926865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:02.927948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:02.928414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:02.928927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] PATH_INFO: `/v3/users/4ed98b05cbf64b55ad8d2443ae1aa344` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:02.931105 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed98b05cbf64b55ad8d2443ae1aa344)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:02.933430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: WARNING py.warnings [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:34:02.933430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: warnings.warn(msg) Aug 29 21:34:02.933430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: Aug 29 21:34:02.935600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:03.022283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f25cba-6ee6-4bfc-8677-3baa3f528c8b service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:03.034284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:03.037547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.037768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.038111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] PATH_INFO: `/v3/users/c26b72a7bb864d188dc600e8edd52ed5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.041761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26b72a7bb864d188dc600e8edd52ed5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:03.044152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:03.054232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed98b05cbf64b55ad8d2443ae1aa344'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.068730 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed98b05cbf64b55ad8d2443ae1aa344'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.093144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:03.107092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed98b05cbf64b55ad8d2443ae1aa344'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.109487 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed98b05cbf64b55ad8d2443ae1aa344'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.118004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:03.162915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e7a77ece-944d-4c7b-be9d-fc385909636e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed98b05cbf64b55ad8d2443ae1aa344'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.172265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26b72a7bb864d188dc600e8edd52ed5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.183356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 820/1639] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:02 2022] DELETE /identity/v3/users/4ed98b05cbf64b55ad8d2443ae1aa344 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:03.186069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9dd7b-b74c-47a3-8a86-9044ad4ddb1e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.186488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9dd7b-b74c-47a3-8a86-9044ad4ddb1e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.186907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9dd7b-b74c-47a3-8a86-9044ad4ddb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.200989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26b72a7bb864d188dc600e8edd52ed5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.224043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26b72a7bb864d188dc600e8edd52ed5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.225784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26b72a7bb864d188dc600e8edd52ed5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.226244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-90f67763-7159-43a1-b549-ac8e6a5ed919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26b72a7bb864d188dc600e8edd52ed5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:03.235736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2cc9dd7b-b74c-47a3-8a86-9044ad4ddb1e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:03.238834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2cc9dd7b-b74c-47a3-8a86-9044ad4ddb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.260026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 820/1640] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:02 2022] DELETE /identity/v3/users/c26b72a7bb864d188dc600e8edd52ed5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:03.262352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bf85-e1bf-4c82-b276-fb344ad1c394 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.262719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bf85-e1bf-4c82-b276-fb344ad1c394 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.262984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bf85-e1bf-4c82-b276-fb344ad1c394 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.265662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 821/1641] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 29 21:34:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:03.266979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.267257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.267548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.300263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:03.301877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.377874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 821/1642] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:03.377874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af434b-a807-4535-aaa1-968743405a9e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.377874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af434b-a807-4535-aaa1-968743405a9e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.377874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af434b-a807-4535-aaa1-968743405a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.416939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-09af434b-a807-4535-aaa1-968743405a9e None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:03.418138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-09af434b-a807-4535-aaa1-968743405a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.449844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 822/1643] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:03.450891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:03.453384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.565859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 822/1644] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 29 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:03.566868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af434b-a807-4535-aaa1-968743405a9e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:03.569631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-09af434b-a807-4535-aaa1-968743405a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.572408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be7edf-4d05-499c-885c-1c44e7e1663d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:03.582546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af434b-a807-4535-aaa1-968743405a9e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:03.585003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:03.586626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.589188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.589460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:03.589614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.590420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.590946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.591449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.604539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:03.606704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:03.613386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:03.616046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ebcf-0ed2-4076-aa4e-0efd1cbcebbb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:03.633256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 823/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:03.634877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d43a-d73c-4fc9-9ace-b84dc0968b52 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.635323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d43a-d73c-4fc9-9ace-b84dc0968b52 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.637643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4d43a-d73c-4fc9-9ace-b84dc0968b52 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.640454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 824/1646] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 29 21:34:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:03.647242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.647525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.647806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.660547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 823/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:03.684635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:03.686525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.703092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 825/1648] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:03.716739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:03.718217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.810425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ccb80-80e7-41db-9662-9604c20152c9 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:03.821524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:03.822532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.822889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.823330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.838209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:03.840047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:03.849576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:03.857899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 824/1649] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:04.781208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:04.783664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:04.807410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d14f2-e5bd-437e-bb8d-5b6b3b416383 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:04.819277 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:04.820784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:04.821384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:04.821830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] PATH_INFO: `/v3/projects/e3fc6dcea8f34aad91a35e71fb44daac` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:04.823558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fc6dcea8f34aad91a35e71fb44daac)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:04.825851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: WARNING py.warnings [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:34:04.825851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: warnings.warn(msg) Aug 29 21:34:04.825851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: Aug 29 21:34:04.828043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:04.893687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:04.920210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fc6dcea8f34aad91a35e71fb44daac', 'request_id': 'req-27a12088-143d-438c-bf3d-8b83fcc2865f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:04.930555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fc6dcea8f34aad91a35e71fb44daac', 'request_id': 'req-27a12088-143d-438c-bf3d-8b83fcc2865f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:04.940082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fc6dcea8f34aad91a35e71fb44daac', 'request_id': 'req-27a12088-143d-438c-bf3d-8b83fcc2865f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:04.976337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] The token cache is being invalidate because project e3fc6dcea8f34aad91a35e71fb44daac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:04.976653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:04.977533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:04.981477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 826/1650] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:04 2022] DELETE /identity/v3/projects/e3fc6dcea8f34aad91a35e71fb44daac => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:04.985513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:05.000936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d533d4-d1a6-41fa-a497-6ce7577ac12e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:05.121727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:05.122465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:05.122865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:05.123260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] PATH_INFO: `/v3/users/d2a3c5dc8a8642429195ce710839c569` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:05.125638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a3c5dc8a8642429195ce710839c569)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:05.129309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:05.214484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a3c5dc8a8642429195ce710839c569'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.226329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a3c5dc8a8642429195ce710839c569'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.237595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a3c5dc8a8642429195ce710839c569'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.238545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a3c5dc8a8642429195ce710839c569'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.262142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a3c5dc8a8642429195ce710839c569'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.270629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 825/1651] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:04 2022] DELETE /identity/v3/users/d2a3c5dc8a8642429195ce710839c569 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:05.874592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:05.877996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:05.893997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a12088-143d-438c-bf3d-8b83fcc2865f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:06.113297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:06.115043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:06.115322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:06.115601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] PATH_INFO: `/v3/projects/45d62a4c1ecb4a8bbf8c71ca5ba6dbb6` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:06.119652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d62a4c1ecb4a8bbf8c71ca5ba6dbb6)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:06.126212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:06.252439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d62a4c1ecb4a8bbf8c71ca5ba6dbb6', 'request_id': 'req-40d5c540-5ba4-4cfd-94ba-517c21761013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.274530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d62a4c1ecb4a8bbf8c71ca5ba6dbb6', 'request_id': 'req-40d5c540-5ba4-4cfd-94ba-517c21761013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.290345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d62a4c1ecb4a8bbf8c71ca5ba6dbb6', 'request_id': 'req-40d5c540-5ba4-4cfd-94ba-517c21761013'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.319963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] The token cache is being invalidate because project 45d62a4c1ecb4a8bbf8c71ca5ba6dbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:06.319963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.327126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.329539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 827/1652] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:05 2022] DELETE /identity/v3/projects/45d62a4c1ecb4a8bbf8c71ca5ba6dbb6 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:06.339246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:06.341668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:06.399681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f99a9-43db-4dfe-a5d6-adbe244d4f09 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:06.630157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:06.630896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:06.631183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:06.631513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] PATH_INFO: `/v3/users/a63f6d28100349598e805d738264e4df` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:06.632802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63f6d28100349598e805d738264e4df)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:06.636757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:06.701117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:06.802831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63f6d28100349598e805d738264e4df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.818705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63f6d28100349598e805d738264e4df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.847227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63f6d28100349598e805d738264e4df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.848310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63f6d28100349598e805d738264e4df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.917944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8dc0fae8-1a41-4c80-91c6-d5f96b4ffc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63f6d28100349598e805d738264e4df'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.926373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 826/1653] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:06 2022] DELETE /identity/v3/users/a63f6d28100349598e805d738264e4df => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:07.218325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:07.226490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:07.248387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5c540-5ba4-4cfd-94ba-517c21761013 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:07.368432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:07.414029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:07.414276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:07.414526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:07.414797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] PATH_INFO: `/v3/projects/6a5291ee2b604a089ae9ecf52210d556` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:07.417508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5291ee2b604a089ae9ecf52210d556)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:07.430566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:07.596675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5291ee2b604a089ae9ecf52210d556', 'request_id': 'req-aae66166-f968-4281-8f87-0c1849712e14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.615380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5291ee2b604a089ae9ecf52210d556', 'request_id': 'req-aae66166-f968-4281-8f87-0c1849712e14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.629105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5291ee2b604a089ae9ecf52210d556', 'request_id': 'req-aae66166-f968-4281-8f87-0c1849712e14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.652134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] The token cache is being invalidate because project 6a5291ee2b604a089ae9ecf52210d556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:07.652862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.656734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-aae66166-f968-4281-8f87-0c1849712e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.671237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 828/1654] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:07 2022] DELETE /identity/v3/projects/6a5291ee2b604a089ae9ecf52210d556 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:07.671571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:07.672713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:07.673258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:07.733620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:07.735984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:07.944020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 827/1655] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:07.954252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:07.954538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:07.954812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.073199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:08.074430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:08.122134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 829/1656] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:08.126820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.151770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef0a62-db69-4ac5-90ec-63f7cfb5e29d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.184323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.185005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.185286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.185563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.187018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:08.187018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.194137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.238509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 828/1657] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:08.240090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.262687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8b7d3-7aa8-40e7-bab7-6cda9f1458a1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.290964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.293896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.296074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.296520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.297372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.307258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.399666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.409869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 830/1658] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:08.413850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8ae1-c156-4e89-924b-f545af2da1e1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.577278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.578311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.578844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.583470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.588895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.591184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.638399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.650175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 829/1659] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:08.657538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fcb0c-171f-41b8-a5cb-9a38f2b76893 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.672023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.674526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.675344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.675832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.676779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.678683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.742549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 831/1660] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:08.744354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.760480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b015e6-64a7-4da0-8906-cc70c9a0b6ff admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.783417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.784769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.785256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.786131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.787562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.789490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee6485-5cfd-404f-997d-c635bff74be8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.815639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 830/1661] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:08.817960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.834287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92715ab-2839-42dc-9ca7-51bec9c19e2c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.847111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.848192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.848192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.849057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] PATH_INFO: `/v3/projects/34f1dc4ece6a47b4b967cbcbce6a2b3a/users/deb9609b62914b5da79255352ee6cfc4/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.879243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f1dc4ece6a47b4b967cbcbce6a2b3a, user_id=deb9609b62914b5da79255352ee6cfc4, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.883030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.917804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 832/1662] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:08 2022] PUT /identity/v3/projects/34f1dc4ece6a47b4b967cbcbce6a2b3a/users/deb9609b62914b5da79255352ee6cfc4/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:08.928008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.928326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.928797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.942970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:08.949661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.952762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.010639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] MFA Rules not processed for user `deb9609b62914b5da79255352ee6cfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:09.013241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.198526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd1ed-7f99-48a5-8937-4e3e420d31d3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.216216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.222029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.227592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.229003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] PATH_INFO: `/v3/projects/d5e4357d29684686b438d150fcbb5ccf` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.237606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e4357d29684686b438d150fcbb5ccf)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.245795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.261698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 831/1663] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:09.262969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.300619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4ed34-b77a-437d-8020-4aa3aa819cbe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.317898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.318717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:09.319609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.320319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.320947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.322323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.326669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.386845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 832/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:09.387892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e4357d29684686b438d150fcbb5ccf', 'request_id': 'req-94bc6024-a829-45de-910c-b1c67be38de8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:09.388931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.406247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e4357d29684686b438d150fcbb5ccf', 'request_id': 'req-94bc6024-a829-45de-910c-b1c67be38de8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:09.420441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e4357d29684686b438d150fcbb5ccf', 'request_id': 'req-94bc6024-a829-45de-910c-b1c67be38de8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:09.447989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] The token cache is being invalidate because project d5e4357d29684686b438d150fcbb5ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:09.448303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:09.453486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-94bc6024-a829-45de-910c-b1c67be38de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:09.463587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 833/1665] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:08 2022] DELETE /identity/v3/projects/d5e4357d29684686b438d150fcbb5ccf => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:09.468796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc3f6-c964-4d46-a5c9-24eafe22ed60 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.469092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc3f6-c964-4d46-a5c9-24eafe22ed60 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.469367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fc3f6-c964-4d46-a5c9-24eafe22ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.515901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860ac2e-1ab8-438c-b341-cef2000204cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.522319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.523932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.524444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.525148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.526145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.528951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.529975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1e1fc3f6-c964-4d46-a5c9-24eafe22ed60 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:09.532110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1e1fc3f6-c964-4d46-a5c9-24eafe22ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.573826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 834/1666] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:09.581563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.581905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.582187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.590519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 833/1667] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/users => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:09.597602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.605761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.656997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b17c3-55f1-42e0-95b9-4e87616a3a18 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.667665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:09.675044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.680079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.684114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.685420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.685714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.686432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.688085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.706180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 835/1668] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:09.712487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.738088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 834/1669] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:09.753867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.756709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f8dd6-a2a5-4c38-b1ad-bdd3db1cfb91 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.772417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.776046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.776336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.778857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.779647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:09.782431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.784516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40cdfe-5505-440b-b247-abdd0e2c90b8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.789695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.802213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.806525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.810316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.810611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] PATH_INFO: `/v3/projects/6a7265c63c6847f18e32f92e28811863/users/9ce16015ec7b4c4ca7a1132912239850/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.864617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 836/1670] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:09.870764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.892023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7265c63c6847f18e32f92e28811863, user_id=9ce16015ec7b4c4ca7a1132912239850, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.907156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.908488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbca64-7243-4718-9d3b-bb559ddc65e5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.925267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.928109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.929687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.929980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.930501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 835/1671] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:09 2022] PUT /identity/v3/projects/6a7265c63c6847f18e32f92e28811863/users/9ce16015ec7b4c4ca7a1132912239850/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:09.934453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.935641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.939368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.003349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:10.025208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 837/1672] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:10.031359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.084215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936dcbd-7181-47ac-9b59-2e9ed699a796 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.258791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d554623-c428-4296-9868-e77d4b4e718c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.261917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.263042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.263571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.264165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.265351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.267670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.278172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.279957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.280619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.280726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.281343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.283638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.304800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 836/1673] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:10.305751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.332152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0a54f-4a0f-4df5-9fa9-b5cc5453a47b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.342554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 838/1674] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:10 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:10.343873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.352550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.353269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.353553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.353907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] PATH_INFO: `/v3/projects/6a7265c63c6847f18e32f92e28811863/users/9ce16015ec7b4c4ca7a1132912239850/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.362482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7265c63c6847f18e32f92e28811863, user_id=9ce16015ec7b4c4ca7a1132912239850, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.365826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932bcd8-d903-4aba-9757-0ec1c4ebaf86 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.374885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb642-fa99-47e8-941f-f00be75dec2e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.391888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 837/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:10 2022] PUT /identity/v3/projects/6a7265c63c6847f18e32f92e28811863/users/9ce16015ec7b4c4ca7a1132912239850/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:10.403672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.403973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.404228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.410233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.410972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.411901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.411901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.412239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.414573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.440222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 839/1676] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:10.450486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.473633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052dce8-2bfa-4fcf-ab64-cb32111d93b1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.482634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] MFA Rules not processed for user `9ce16015ec7b4c4ca7a1132912239850`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:10.483896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:10.591684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:10.700420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.701101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.701520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.701796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.702469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.704087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.747992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 838/1677] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:10.764016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.764680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 840/1678] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:10.766199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:10.785568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.827304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fef23-fc68-4737-bb59-e216d40f13b8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.855623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.859910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.859910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.859910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] PATH_INFO: `/v3/projects/a1b1cdd9480e48a6ab348e45715ef60d/users/ff1f29795cba4dc287a110ede9a5f028/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.909773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b1cdd9480e48a6ab348e45715ef60d, user_id=ff1f29795cba4dc287a110ede9a5f028, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.919403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a664f95-33ad-4f75-8cf6-4e9870d490c4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.954046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:10.957020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 841/1679] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:10 2022] PUT /identity/v3/projects/a1b1cdd9480e48a6ab348e45715ef60d/users/ff1f29795cba4dc287a110ede9a5f028/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:10.967544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.967832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.968130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:11.003095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc4395-b751-40a9-a08e-2485e030f18d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:11.027955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] MFA Rules not processed for user `ff1f29795cba4dc287a110ede9a5f028`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:11.029218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:11.161518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:11.162530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:11.162815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:11.163091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:11.216055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 842/1680] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:11.221803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:11.409716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:11.412337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:11.447008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 839/1681] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:11.521815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:11.521815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:11.558022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771de03-d0f9-4c98-bca3-c47c6d1403c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:11.563737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7fcfa-0fe7-42f9-b2b2-6d1506daee74 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:11.571139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:11.572214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:11.572214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:11.572417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:11.572949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:11.593337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:11.594338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:11.598377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:11.601821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:11.602328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] PATH_INFO: `/v3/users/6f1bdcc244f9484f8aca8f5e59839027` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:11.615191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1bdcc244f9484f8aca8f5e59839027)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:11.617484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:11.687578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 843/1682] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:11 2022] POST /identity/v3/projects => generated 365 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:11.688522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:11.792949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:11.802302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1bdcc244f9484f8aca8f5e59839027'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:11.832278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1bdcc244f9484f8aca8f5e59839027'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:11.847189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30fc5b-fe9c-4116-8cf9-4df1f75cd986 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:11.853427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1bdcc244f9484f8aca8f5e59839027'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:11.854574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1bdcc244f9484f8aca8f5e59839027'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:11.867839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:11.868533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:11.868830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:11.869174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:11.869890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:11.873677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:11.911471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 844/1683] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:11 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:11.917119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1bdcc244f9484f8aca8f5e59839027'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:11.918366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:11.923346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:11.935540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 840/1684] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:11 2022] DELETE /identity/v3/users/6f1bdcc244f9484f8aca8f5e59839027 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:11.936696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:11.940749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:11.955883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b19f5-a1f7-4063-be45-a1304c60d1d2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.076556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:12.105670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:12.106399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.106745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.107124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] PATH_INFO: `/v3/users/deb9609b62914b5da79255352ee6cfc4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.109072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb9609b62914b5da79255352ee6cfc4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:12.114606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:12.260182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7602-501b-4c54-9669-13e2a3e244a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.287072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:12.287914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.288261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.288541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.289231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:12.293424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2e3ed-08ee-4886-bac3-f0debbdf86c3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:12.304152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb9609b62914b5da79255352ee6cfc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:12.341274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 841/1685] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:12.342791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3efd41-ffba-4ca2-b192-1eb7d39a9cc3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.342917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3efd41-ffba-4ca2-b192-1eb7d39a9cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.343158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3efd41-ffba-4ca2-b192-1eb7d39a9cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.353740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb9609b62914b5da79255352ee6cfc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:12.374348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb9609b62914b5da79255352ee6cfc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:12.375124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb9609b62914b5da79255352ee6cfc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:12.377096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb9609b62914b5da79255352ee6cfc4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:12.412336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3e3efd41-ffba-4ca2-b192-1eb7d39a9cc3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:12.413964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3e3efd41-ffba-4ca2-b192-1eb7d39a9cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.425504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 845/1686] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:11 2022] DELETE /identity/v3/users/deb9609b62914b5da79255352ee6cfc4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:12.425504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:12.429463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.557911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:12.595921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 842/1687] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:12.597883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.598211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.598540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.653661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d2195-a6b7-4a2e-aa84-24d0fe5fea8a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.661707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:12.664244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.670220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:12.671285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.671703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.672156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] PATH_INFO: `/v3/projects/f8e9d80d0e874241ae74b5a97260b97a/users/604bfa497e654652869c0fcac0064489/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.697970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 843/1688] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:12.726567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e9d80d0e874241ae74b5a97260b97a, user_id=604bfa497e654652869c0fcac0064489, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:12.728953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:12.731966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.740443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:12.799076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 846/1689] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:12 2022] PUT /identity/v3/projects/f8e9d80d0e874241ae74b5a97260b97a/users/604bfa497e654652869c0fcac0064489/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:12.801926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:12.809076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.931340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311296-8e87-4332-ae6d-30bed3daf601 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.968953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ca6f-1d47-4361-a811-217e3c3971e8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.983632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:12.984908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.985287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.990370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.092754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.096058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.096333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.099927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.179596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:13.213802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.219150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.255661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 847/1690] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:13.256846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:13.279908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320010-b223-4ca9-8c5c-4873e3eb055e service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:13.294822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.297371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.297836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.298687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.300789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.303388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.305591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.307538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23d8fc-cf0f-439f-9ef9-aad12b5c7ccf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.337511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 848/1691] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:13.339208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:13.340363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 844/1692] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:13.353077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b58f7-963e-475c-aeb4-da0f69a70230 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:13.359131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.360105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.360748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.361173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] PATH_INFO: `/v3/projects/f8e9d80d0e874241ae74b5a97260b97a/users/604bfa497e654652869c0fcac0064489/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.363403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e9d80d0e874241ae74b5a97260b97a, user_id=604bfa497e654652869c0fcac0064489, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.367010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.383726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 849/1693] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:13 2022] PUT /identity/v3/projects/f8e9d80d0e874241ae74b5a97260b97a/users/604bfa497e654652869c0fcac0064489/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:13.393886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.394625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.396292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.440313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] MFA Rules not processed for user `604bfa497e654652869c0fcac0064489`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:13.442677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:13.612239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 845/1694] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:13.616713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:13.618746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:13.708930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:13.729861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e526d9-d7a4-4ea8-b911-2a02892d7622 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:13.735439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.736894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.737300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.738715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.751041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.752784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.780276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 850/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:14.167745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.206798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c719ff-7f96-4561-a21a-fe7a2ef182a5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.225489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.226678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.226678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.227027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.234943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:14.582177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.584244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e913fd6-b361-4583-864d-532af04fd3f7 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.611905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 846/1696] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:15.582114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:15.589438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:15.825050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a7a29-6595-47ce-b2ec-2a7896d465c3 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:15.832003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:15.833231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:15.833587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:15.834082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] PATH_INFO: `/v3/projects/23e5e8ff1d1f45a1a9d1f01889f496b1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:15.839617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e5e8ff1d1f45a1a9d1f01889f496b1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:15.841572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:15.925849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e5e8ff1d1f45a1a9d1f01889f496b1', 'request_id': 'req-d646158c-9091-4fce-977d-3b2f28f05542'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.947591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e5e8ff1d1f45a1a9d1f01889f496b1', 'request_id': 'req-d646158c-9091-4fce-977d-3b2f28f05542'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.959539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e5e8ff1d1f45a1a9d1f01889f496b1', 'request_id': 'req-d646158c-9091-4fce-977d-3b2f28f05542'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.998767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] The token cache is being invalidate because project 23e5e8ff1d1f45a1a9d1f01889f496b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:15.999124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.999853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d646158c-9091-4fce-977d-3b2f28f05542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.007271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.007880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.007880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.008015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 851/1697] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:15 2022] DELETE /identity/v3/projects/23e5e8ff1d1f45a1a9d1f01889f496b1 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:16.053731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:16.055590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.122837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:16.210860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.211458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.211880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.220166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 847/1698] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:16.256387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:16.259880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.279183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.289326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 852/1699] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:16.305614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d259b-c43d-4edd-b25c-4b7a99e0fb03 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.311971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.312617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.312916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.313192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.314578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:16.314883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.316983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.334386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 848/1700] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:16.338891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.362487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd83b-0b24-4152-8499-16ecb2a103f9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.386908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.396381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.396697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.396972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.397655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.409056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.498934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 853/1701] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:16.502807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.517913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc2a8b-5e10-420e-9252-15ed52380fa7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.550186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.552163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.805305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:16.844480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.845136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.845439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.845717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.846369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.848394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810bf95-4665-4318-84ee-f19069686c1a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.864545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969b2-d339-4445-8dff-a422a3020f05 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.901633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.902522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.904271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.904896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 849/1702] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/users => generated 287 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:16.905586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] PATH_INFO: `/v3/projects/34f1dc4ece6a47b4b967cbcbce6a2b3a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.909741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f1dc4ece6a47b4b967cbcbce6a2b3a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.910030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.910359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.910626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.913356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.051667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f1dc4ece6a47b4b967cbcbce6a2b3a', 'request_id': 'req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.068325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f1dc4ece6a47b4b967cbcbce6a2b3a', 'request_id': 'req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.068876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] MFA Rules not processed for user `5344cf7b391d44beac2c602ac3e2e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:17.072006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.077325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f1dc4ece6a47b4b967cbcbce6a2b3a', 'request_id': 'req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.112729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] The token cache is being invalidate because project 34f1dc4ece6a47b4b967cbcbce6a2b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:17.113133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.116249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1a8b78a8-9eb3-45f8-92fb-a442d8a3b580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.130560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 854/1703] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:16 2022] DELETE /identity/v3/projects/34f1dc4ece6a47b4b967cbcbce6a2b3a => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:17.132783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.133086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.133390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.246052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] MFA Rules not processed for user `89b272427c6444b49293e5046c686ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:17.247366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.392650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 850/1704] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:17.400510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.403068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.512637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:17.515953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 855/1705] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:17.519019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.522708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.592236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c16182-5b51-4f14-8d09-664cae224a9d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.608657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.609411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.609697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.610085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.610750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.612406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a727b-33a6-4f67-ba14-89c332ad5b23 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.649962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c42f3-1e36-4d39-acba-2e36784fc637 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.651402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 851/1706] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:17.652841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.653333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.653798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.666979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.668144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.668947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.670049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] PATH_INFO: `/v3/users/9ce16015ec7b4c4ca7a1132912239850` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.672276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce16015ec7b4c4ca7a1132912239850)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.677823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.694502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:17.722582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] MFA Rules not processed for user `5344cf7b391d44beac2c602ac3e2e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:17.723964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.761932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 852/1707] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:17.763364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.765831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.934171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce16015ec7b4c4ca7a1132912239850'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.991273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce16015ec7b4c4ca7a1132912239850'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.000893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716b30-f84d-4ff1-90a2-41b5280b1a24 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.009497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.011216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.012299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.012525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.016309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce16015ec7b4c4ca7a1132912239850'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.018525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce16015ec7b4c4ca7a1132912239850'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.020048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce16015ec7b4c4ca7a1132912239850'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.024731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.026788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.042145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 853/1708] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:18.043196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.058244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1a8ce-e8a3-43bc-b49e-749787ad4dbc service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.063792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.064950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.065413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.065924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.067886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.068815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 856/1709] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:17 2022] DELETE /identity/v3/users/9ce16015ec7b4c4ca7a1132912239850 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:18.073650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.074074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.087093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.090117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 854/1710] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:18.104485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.120155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.144899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64411-d208-46d8-9ca8-1898d92235bd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.372866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.373999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.374479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.374947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] PATH_INFO: `/v3/projects/dea2523a77b345aa8d406f725466d9fa/users/36d5140e4a064d6bb2510f12f5eaff22/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.414705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d741212-a4ae-498d-ab3b-20cfce809573 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.446344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.448170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.448830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.449114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.470987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.472109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea2523a77b345aa8d406f725466d9fa, user_id=36d5140e4a064d6bb2510f12f5eaff22, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.496602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffd64d-7873-4189-ac68-a729ed8e28b3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.585857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 855/1711] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:18 2022] PUT /identity/v3/projects/dea2523a77b345aa8d406f725466d9fa/users/36d5140e4a064d6bb2510f12f5eaff22/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:18.599131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3543-24ee-46c6-9213-73a409d2643f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.599407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3543-24ee-46c6-9213-73a409d2643f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.599694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3543-24ee-46c6-9213-73a409d2643f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.742893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-55ca3543-24ee-46c6-9213-73a409d2643f None None] MFA Rules not processed for user `36d5140e4a064d6bb2510f12f5eaff22`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:18.744150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-55ca3543-24ee-46c6-9213-73a409d2643f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.820464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.827782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.868915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:18.886634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 857/1712] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:18.889644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.914653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.969738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1670be3-13e1-4aec-a455-ff701cf176b3 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:19.025687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:19.063825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 856/1713] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:19.085656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59378e-fb4b-4645-ab26-a406d6d6528c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.086905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59378e-fb4b-4645-ab26-a406d6d6528c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.095091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59378e-fb4b-4645-ab26-a406d6d6528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.148472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:19.148472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.148913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.149373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] PATH_INFO: `/v3/projects/6a7265c63c6847f18e32f92e28811863` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.151950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7265c63c6847f18e32f92e28811863)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:19.163462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:19.209634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3c59378e-fb4b-4645-ab26-a406d6d6528c None None] MFA Rules not processed for user `b8ac60b941b04b2eb01281ba5f0e0b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:19.211159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3c59378e-fb4b-4645-ab26-a406d6d6528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.308423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7265c63c6847f18e32f92e28811863', 'request_id': 'req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.323939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7265c63c6847f18e32f92e28811863', 'request_id': 'req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.332026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7265c63c6847f18e32f92e28811863', 'request_id': 'req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.350398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] The token cache is being invalidate because project 6a7265c63c6847f18e32f92e28811863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:19.350752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.351673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.356965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 858/1714] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:18 2022] DELETE /identity/v3/projects/6a7265c63c6847f18e32f92e28811863 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:19.357897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:19.359500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.385275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 857/1715] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:19.386617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251e9a-3deb-45c2-887a-ca3531002f2d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.386975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251e9a-3deb-45c2-887a-ca3531002f2d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.387322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251e9a-3deb-45c2-887a-ca3531002f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.454527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7f251e9a-3deb-45c2-887a-ca3531002f2d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:19.457637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7f251e9a-3deb-45c2-887a-ca3531002f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.586106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b43c09-92a8-4bbc-a77d-39eb24cdc15c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:19.603081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:19.603994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.604305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.604777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.608255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.638635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 858/1716] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:19.640197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.642551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.642960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.699807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] MFA Rules not processed for user `b8ac60b941b04b2eb01281ba5f0e0b96`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:19.700980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.764888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:19.810161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:19.814033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d7aad-cc95-43c5-9588-951d78d7535d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:19.861421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 859/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:19.869504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.869847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.870268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.884648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:19.912254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 859/1718] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:19.975305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:19.989149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:20.006720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.017471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 860/1719] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:20.023439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.028562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba716afc-f760-4284-914a-4ad0e8362ca2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.050012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.050659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.050987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.051436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.066068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f21b8-c747-4c18-93e9-43ab3b7608f3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.077837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.085539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.087007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.087712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.087996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.088272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.091503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:20.091837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.099820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.119387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 860/1720] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:20.152793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 861/1721] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:20.156711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.179294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e593c0-927b-4b6c-903f-b1979ab4acc7 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.189521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.190436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.191370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.191790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.192508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.197362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.273980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 861/1722] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:20.279100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.294348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d409a2-df5a-4963-9c4b-c3325f4ad1a6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.522611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.523251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.523565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.523850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.524524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.528768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387af7f5-5030-4f91-92de-5f6a502b1570 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.575700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 862/1723] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/users => generated 290 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:20.581169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.608703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371c1b2-d5d5-4811-9de7-aee3ffe47357 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.634661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.636544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.637032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.637414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.639297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.646689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ad15f-759f-4f5d-aec5-c342a78f8708 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.681291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00525b2-4026-4eed-a35e-4015ad23f38d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.681575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00525b2-4026-4eed-a35e-4015ad23f38d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.682053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00525b2-4026-4eed-a35e-4015ad23f38d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.684241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 863/1724] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 29 21:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:34:20.686130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 862/1725] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:20.689727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00525b2-4026-4eed-a35e-4015ad23f38d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.704001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.704001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.704001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.739311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00525b2-4026-4eed-a35e-4015ad23f38d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.757055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.757572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.758023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.758304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.758971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.760653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.814635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 864/1726] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:20.815653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.834165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:20.842868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:20.847293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d94923-d377-40a3-9c84-b5564ca1ea25 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.874523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.875945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.876388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.876766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] PATH_INFO: `/v3/projects/364b1c4673f4491295b378856bee5343/users/d778f672e6454f96a37a31169eb409fa/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.942126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=364b1c4673f4491295b378856bee5343, user_id=d778f672e6454f96a37a31169eb409fa, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.952691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba306c85-48a5-4ed1-90b4-78d6b56c84ca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.022088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 865/1727] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:20 2022] PUT /identity/v3/projects/364b1c4673f4491295b378856bee5343/users/d778f672e6454f96a37a31169eb409fa/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:21.023864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.024154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.024434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.059893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 863/1728] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 29 21:34:21.108391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] MFA Rules not processed for user `d778f672e6454f96a37a31169eb409fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:21.109847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:21.125183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.127170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:21.157160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84142039-94e4-4938-ae93-6998f900ff90 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.372453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.373157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.373439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.373714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.394288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 866/1729] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:21.398429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.400791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.403158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.447667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 864/1730] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:21.452383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:21.602027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f5451-345f-4538-8fb8-9aa73363dd48 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.616631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.617209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.617564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.617971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.618878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.621303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755718e-8284-440f-8180-ac7d85936446 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.736039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 867/1731] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/projects => generated 377 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:21.741561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.766019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72dd5-88b5-4760-8c0e-5862c769d45c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.882965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df652b-1017-4a2a-bc65-f5b64ff8f7fe None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.883524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df652b-1017-4a2a-bc65-f5b64ff8f7fe None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.883948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df652b-1017-4a2a-bc65-f5b64ff8f7fe None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.885564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 868/1732] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 29 21:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:21.895908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.896429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.896874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.925380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:21.931764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:21.933849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:21.944516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.945340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.945725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.946368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.947632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.951322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.999977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 865/1733] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:22.001504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.038215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ba844-4728-4e11-951b-ff40aba83477 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.047214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.048729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.049330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.049956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.051864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.055496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.082485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 866/1734] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:22.084070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.104129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085d546-a1aa-43cb-8e9d-2bef47603cc9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.113364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.115081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.115577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.116161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.117249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.119064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.138188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 869/1735] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:22.141316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.142804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 867/1736] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:22.144422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.168118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020be3cf-d65c-4dfc-91e4-aee4e90ecf6f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.169886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8637cc9-98a4-40e9-98c4-0a58e517fc60 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.179590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.180379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.180945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.181326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] PATH_INFO: `/v3/projects/74196d7a38ef438e9a1b1b6bbf5076cd/users/d6e6a19f29714ba8984c07758391cf64/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.181546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.182739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.183311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.186027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.208994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.213169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900d0d3-1bde-4882-9296-ef7848bd6ca6 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.218241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74196d7a38ef438e9a1b1b6bbf5076cd, user_id=d6e6a19f29714ba8984c07758391cf64, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.228097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 868/1737] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:22.230373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.298188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 870/1738] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:22 2022] PUT /identity/v3/projects/74196d7a38ef438e9a1b1b6bbf5076cd/users/d6e6a19f29714ba8984c07758391cf64/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:22.318157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.318743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.319347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.436940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] MFA Rules not processed for user `d6e6a19f29714ba8984c07758391cf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:22.438378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:22.612873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.615337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:22.644629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 869/1739] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:22.651656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.816190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011d60-c484-4485-83ba-de6f82f2583a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.843237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.844222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.844747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.845498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.849238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:22.850514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07ac2-6c3e-461f-ad9d-17e772d5ee78 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:22.867949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:22.868943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.869437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.869955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.870846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.873116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.881328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:22.884697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d90abf-64c4-4112-8b84-19d68d3cd149 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:22.911075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 871/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:22.949431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:22.949864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:22.950244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:22.963960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 870/1741] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:22 2022] POST /identity/v3/projects => generated 379 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:22.965245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:22.999418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] MFA Rules not processed for user `5344cf7b391d44beac2c602ac3e2e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:23.001090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:23.028198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:23.029121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:23.122889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 872/1742] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:23.142428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f87997-b5b2-4866-92ef-352938b60ad6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.148932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.150355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.150866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.151339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.152198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.154795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:23.193145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 871/1743] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:22 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:23.198620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:23.216366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76507e-6d1a-4e87-ab75-cf333159f362 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.224726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.225809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.227586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.227956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.229072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.231121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:23.243267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 873/1744] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:23.247739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:23.272293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57286-7411-42a8-9ca6-a4fae85c9416 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.287332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.288289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.288289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.288560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] PATH_INFO: `/v3/projects/4286889c73eb4a848c46f81449e062d6/users/729346a7cb634d8789c929c1e470775f/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.386774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4286889c73eb4a848c46f81449e062d6, user_id=729346a7cb634d8789c929c1e470775f, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.389782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:23.450156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:23.452063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:23.459378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 872/1745] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:23 2022] PUT /identity/v3/projects/4286889c73eb4a848c46f81449e062d6/users/729346a7cb634d8789c929c1e470775f/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:23.466147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:23.499365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8eac9-b0c0-4a7c-9fe9-7f63bcdcdb2a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.657846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.658552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.658835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.659109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] PATH_INFO: `/v3/users/36d5140e4a064d6bb2510f12f5eaff22` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.661079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d5140e4a064d6bb2510f12f5eaff22)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.665694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:23.676821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712ccc-c8a0-4c6c-939d-1db8a1b8f24e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.688172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.689379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.689499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.690085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.690942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.693172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce509e-74e9-4209-bdc7-bf223ea0a266 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:23.753516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 873/1746] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:23.757228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b714a-e48b-46f8-9e9c-68f394450eb1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.757228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b714a-e48b-46f8-9e9c-68f394450eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.757228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b714a-e48b-46f8-9e9c-68f394450eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.890181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d5140e4a064d6bb2510f12f5eaff22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:23.942956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a45b714a-e48b-46f8-9e9c-68f394450eb1 None None] MFA Rules not processed for user `55a5c6359109440fbb66d985fafc24bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:23.945028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a45b714a-e48b-46f8-9e9c-68f394450eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:23.953696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d5140e4a064d6bb2510f12f5eaff22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:23.986889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d5140e4a064d6bb2510f12f5eaff22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:23.988758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d5140e4a064d6bb2510f12f5eaff22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:23.992339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d5140e4a064d6bb2510f12f5eaff22'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.058396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 874/1747] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:23 2022] DELETE /identity/v3/users/36d5140e4a064d6bb2510f12f5eaff22 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:24.065690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:24.073964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.146094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:24.230645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f170450-6814-41c7-9668-e87fb38a0d0d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:24.239578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:24.240269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.240548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.240863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] PATH_INFO: `/v3/projects/4286889c73eb4a848c46f81449e062d6/users/729346a7cb634d8789c929c1e470775f/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.248294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4286889c73eb4a848c46f81449e062d6, user_id=729346a7cb634d8789c929c1e470775f, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:24.251036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bb2fe-4e8e-478b-9659-72ba127d094c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:24.258075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 874/1748] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:24.261439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e043-c36f-4474-a789-3954367f9d27 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.262190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e043-c36f-4474-a789-3954367f9d27 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.263019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e043-c36f-4474-a789-3954367f9d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.284950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 875/1749] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:24 2022] PUT /identity/v3/projects/4286889c73eb4a848c46f81449e062d6/users/729346a7cb634d8789c929c1e470775f/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:24.286411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.286701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.286973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.334001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:24.335613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.367392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:24.401740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fe31e043-c36f-4474-a789-3954367f9d27 None None] MFA Rules not processed for user `6f0d97b196f44ec985db30f86d4ad3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:24.403010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fe31e043-c36f-4474-a789-3954367f9d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.476986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 876/1750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:24.478286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:24.484158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.655863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 875/1751] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:24.657627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.657894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.661720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.736351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] MFA Rules not processed for user `729346a7cb634d8789c929c1e470775f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:24.737620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.796061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e745-fdd1-4d67-b53c-b3eb6af18f3b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:24.837743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:24.841348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.842131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.845424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.005425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 876/1752] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:25.005425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.011196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.015183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:25.025135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.043647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:25.093836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 877/1753] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:25.094857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.100023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:25.194846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c958c8-b04d-48ac-a7d7-db69751b23a9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.199814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.200653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.200990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.201646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.257867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.259413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.323281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 877/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:25.352926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.430707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1c03f-f916-44a2-be86-d17a6958a28a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.458253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.462329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.462649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.480087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.507469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1483586-329b-474c-ab32-6c16d154707d service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.528416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.531408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.536252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.540335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.542074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.555531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.607864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.628426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c78fe5-82b3-407b-b592-77d6506cd95a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.650100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 878/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:25.750467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 878/1756] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:25.856280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.858257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:25.890276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1e101-7a0d-4de0-9bb7-05cf0fe3aaf0 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.906131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.906767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.907054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.907313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.912415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:25.950489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:26.049323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f835d-2d81-4d78-ac0d-ae21c386c72a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.049699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f835d-2d81-4d78-ac0d-ae21c386c72a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.050025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f835d-2d81-4d78-ac0d-ae21c386c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.116746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0c8f835d-2d81-4d78-ac0d-ae21c386c72a None None] MFA Rules not processed for user `89b272427c6444b49293e5046c686ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:26.120427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0c8f835d-2d81-4d78-ac0d-ae21c386c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:26.141498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:26.150492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:26.191971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 879/1757] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:26.340361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 879/1758] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:27.236703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:27.238878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:27.303148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61030f-3d76-4370-b1df-f151c9f9585f service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:27.331849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:27.337949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.341312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.343322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] PATH_INFO: `/v3/projects/dea2523a77b345aa8d406f725466d9fa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.348081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea2523a77b345aa8d406f725466d9fa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.359418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.506027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea2523a77b345aa8d406f725466d9fa', 'request_id': 'req-d364593b-6694-490b-9321-ae90b84c3e1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:27.532356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea2523a77b345aa8d406f725466d9fa', 'request_id': 'req-d364593b-6694-490b-9321-ae90b84c3e1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:27.559559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea2523a77b345aa8d406f725466d9fa', 'request_id': 'req-d364593b-6694-490b-9321-ae90b84c3e1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:27.592194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] The token cache is being invalidate because project dea2523a77b345aa8d406f725466d9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:27.592491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:27.594736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d364593b-6694-490b-9321-ae90b84c3e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:27.600646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 880/1759] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:27 2022] DELETE /identity/v3/projects/dea2523a77b345aa8d406f725466d9fa => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:27.603605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.604211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.604528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.678546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:27.684772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:27.924744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 880/1760] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:27.933746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.934140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.934793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.980875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:27.982426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:28.009319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.016070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 881/1761] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:28.023941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de16b-744f-4102-9fc6-6f85af78faa6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.037497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.038501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.039097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.039574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.040631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:28.041143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.043704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.080781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.088226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 881/1762] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:28.094678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2abdab-066d-4f46-a125-30f666c7597a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.113087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.114363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.114745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.115117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.115866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.117945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d54ee-53eb-4b5f-9faa-9d4a46bde230 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.200396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.201750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 882/1763] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:28 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:28.216756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8ec91-e7c9-4670-9208-ed056cd9275a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.290096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:28.350396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.351056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.351315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.351592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.352258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.354310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.389575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e37e3-b581-41a8-baf2-d0cb0fdd6fc3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.390133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e37e3-b581-41a8-baf2-d0cb0fdd6fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.390133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e37e3-b581-41a8-baf2-d0cb0fdd6fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.431195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 882/1764] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:28 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:28.435617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.459758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c5e41-47d3-4edc-8cc4-7185bcd9554e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.485140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.487406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.488264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.489019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.490100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.491900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.512394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 883/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:28.515964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.532258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade6ed-1f2e-4dae-9657-c4ccdf735c02 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.540079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.541442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.541963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-229e37e3-b581-41a8-baf2-d0cb0fdd6fc3 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:28.548991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.548991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.548991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.550492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.551475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-229e37e3-b581-41a8-baf2-d0cb0fdd6fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:28.588683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 884/1766] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:28.591691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.621982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbae9b5-f230-471e-8aaf-7ac5e7a8f40c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:28.634664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:28.636918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:28.638953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.640939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.641433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] PATH_INFO: `/v3/projects/5fd85b240f3346d7b5daf23c1925a298/users/57ba632ebe8b41a9af86aa48e9865a72/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.685547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 883/1767] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:28.704320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd85b240f3346d7b5daf23c1925a298, user_id=57ba632ebe8b41a9af86aa48e9865a72, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:28.714106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:28.757421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 885/1768] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:28 2022] PUT /identity/v3/projects/5fd85b240f3346d7b5daf23c1925a298/users/57ba632ebe8b41a9af86aa48e9865a72/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:28.770747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:28.771046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:28.771318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:28.847184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:28.849600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:28.856301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] MFA Rules not processed for user `57ba632ebe8b41a9af86aa48e9865a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:28.861523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:29.201471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 884/1769] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:29.205328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:29.208418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:29.383812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf6246-0471-4b15-9518-21a04d778260 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:29.394644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:29.395511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:29.395800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:29.396075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:29.446735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806fe54-531d-4f34-9747-d0cf6cf1699b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:29.457541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:29.458448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:29.462709 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:29.463260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] PATH_INFO: `/v3/users/ff1f29795cba4dc287a110ede9a5f028` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:29.465953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff1f29795cba4dc287a110ede9a5f028)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:29.469849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:29.599283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:29.600959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:29.607303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:29.619671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 886/1770] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:34:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:34:29.621808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:29.646311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0762-5242-4ace-97e5-95edf1ab3fee service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:29.660633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff1f29795cba4dc287a110ede9a5f028'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:29.665994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:29.668077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:29.668375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:29.668680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:29.669415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:29.674576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:29.696347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff1f29795cba4dc287a110ede9a5f028'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:29.713906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff1f29795cba4dc287a110ede9a5f028'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:29.719801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff1f29795cba4dc287a110ede9a5f028'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:29.746342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:29.782141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-75845ecc-02fc-488e-b89b-3737f99a2bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff1f29795cba4dc287a110ede9a5f028'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:29.809676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 887/1771] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:29 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:29.810628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:29.818530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:29.823527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 885/1772] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:29 2022] DELETE /identity/v3/users/ff1f29795cba4dc287a110ede9a5f028 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:29.829439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:29.829936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:29.830350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:29.915854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:29.920261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:30.200030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524bcdb-cb28-48af-a6d5-0382be063563 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:30.217336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 886/1773] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:30.219168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:30.222670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:30.223121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:30.226904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:30.226904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:30.231235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:30.323003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:30.323835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 888/1774] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:29 2022] POST /identity/v3/users => generated 299 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:30.349287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a17d0-fe30-40e0-8fb4-47c3f172d424 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:30.349679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:30.349679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:30.376633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:30.377555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:30.377745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:30.378025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:30.379235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:30.406751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:30.481860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 887/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:30.484685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:30.559570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8e746-7fc0-4980-ab4f-92e144d03087 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:30.567696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f591-24a9-4879-a3c8-568f57aa60ba admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:30.576244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:30.577099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:30.577388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:30.577667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:30.605293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:30.607164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:30.607787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:30.608216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] PATH_INFO: `/v3/projects/604fbf0b460144d6b17048ed8bd5877f/users/224694487bfa45eb82ddfa924f439a87/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:30.633297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:30.643851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:30.665689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604fbf0b460144d6b17048ed8bd5877f, user_id=224694487bfa45eb82ddfa924f439a87, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:30.685713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:30.687419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 889/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:30.742222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 888/1777] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:30 2022] PUT /identity/v3/projects/604fbf0b460144d6b17048ed8bd5877f/users/224694487bfa45eb82ddfa924f439a87/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:30.748672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:30.769153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5ccee-0e36-4d58-b2ed-afef227ad100 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:30.984016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:30.985835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:30.987788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:30.988214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:30.992223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:30.996097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42558a-93ed-461b-8419-f546cb258806 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:31.037100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 890/1778] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:31.041329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:31.060154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f387a-8c2e-463c-9a27-e257963881d5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:31.078371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:31.079059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.079336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.079610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] PATH_INFO: `/v3/projects/604fbf0b460144d6b17048ed8bd5877f/users/224694487bfa45eb82ddfa924f439a87/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.082902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=604fbf0b460144d6b17048ed8bd5877f, user_id=224694487bfa45eb82ddfa924f439a87, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:31.093135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:31.120690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 889/1779] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:31 2022] PUT /identity/v3/projects/604fbf0b460144d6b17048ed8bd5877f/users/224694487bfa45eb82ddfa924f439a87/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:31.132127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed347352-ce43-4be4-bbe0-de032d7293ac None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.133090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed347352-ce43-4be4-bbe0-de032d7293ac None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.133416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed347352-ce43-4be4-bbe0-de032d7293ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.138939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:31.140528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.152718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:31.261212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ed347352-ce43-4be4-bbe0-de032d7293ac None None] MFA Rules not processed for user `224694487bfa45eb82ddfa924f439a87`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:31.261212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ed347352-ce43-4be4-bbe0-de032d7293ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.353284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdc1e7-97fb-4fa0-b1ab-a7ffc7031e2e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:31.366400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:31.368138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.368422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.369760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] PATH_INFO: `/v3/users/d778f672e6454f96a37a31169eb409fa` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.371454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d778f672e6454f96a37a31169eb409fa)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:31.377256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:31.563261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 891/1780] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:31.564738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060c8c-3b7a-49e2-ab38-2f6ce8f91234 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.565032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060c8c-3b7a-49e2-ab38-2f6ce8f91234 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.565309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17060c8c-3b7a-49e2-ab38-2f6ce8f91234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.568637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:31.576921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd778f672e6454f96a37a31169eb409fa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:31.596970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd778f672e6454f96a37a31169eb409fa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:31.622802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd778f672e6454f96a37a31169eb409fa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:31.625494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd778f672e6454f96a37a31169eb409fa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:31.641628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17060c8c-3b7a-49e2-ab38-2f6ce8f91234 None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:31.646013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17060c8c-3b7a-49e2-ab38-2f6ce8f91234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.654407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd778f672e6454f96a37a31169eb409fa'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:31.664182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 890/1781] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:31 2022] DELETE /identity/v3/users/d778f672e6454f96a37a31169eb409fa => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:31.668315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:31.669840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.838711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bd951-e127-4d33-a94a-13773037f62a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:31.844030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:31.844821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.845102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.845378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.847684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.899453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 892/1782] 10.222.0.72 () {58 vars in 1015 bytes} [Mon Aug 29 21:34:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:31.900693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485152-7e6a-452d-b9be-037df8a439af None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:31.900980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485152-7e6a-452d-b9be-037df8a439af None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:31.901311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485152-7e6a-452d-b9be-037df8a439af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:31.944752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-00485152-7e6a-452d-b9be-037df8a439af None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:31.948856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00485152-7e6a-452d-b9be-037df8a439af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:31.991286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:31.992892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:32.019944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 891/1783] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:32.066315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:32.067795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.123978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 893/1784] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:32.306272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276360c-52f6-47cf-9c34-612538ba9c8c service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:32.324276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:32.324987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:32.325232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485152-7e6a-452d-b9be-037df8a439af None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:32.325347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:32.325530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] PATH_INFO: `/v3/users/57ba632ebe8b41a9af86aa48e9865a72` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:32.328763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00485152-7e6a-452d-b9be-037df8a439af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.329324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ba632ebe8b41a9af86aa48e9865a72)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:32.337056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:32.382658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:32.464619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:32.486002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ba632ebe8b41a9af86aa48e9865a72'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:32.499136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485152-7e6a-452d-b9be-037df8a439af None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:32.508372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:32.509081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:32.509360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:32.509649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:32.512491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ba632ebe8b41a9af86aa48e9865a72'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:32.528286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ba632ebe8b41a9af86aa48e9865a72'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:32.531497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ba632ebe8b41a9af86aa48e9865a72'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:32.531850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:32.533339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:32.564102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 894/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:32.565022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:32.567125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.587419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-43c89fbd-770d-491c-9761-7a65b948148f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ba632ebe8b41a9af86aa48e9865a72'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:32.594193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fddf2e-f630-44fb-87d6-da8dc1ebbd31 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:32.611019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:32.611659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:32.611942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:32.612244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:32.616941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.622010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 892/1786] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:32 2022] DELETE /identity/v3/users/57ba632ebe8b41a9af86aa48e9865a72 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:32.626593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:32.626877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:32.627154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:32.646311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:32.647442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:32.675170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 895/1787] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:32.676217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:32.677590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.709992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:32.711028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:32.711149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f84b4-8428-4c5b-81ac-25f537fb5296 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:32.870933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:32.871606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:32.871888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:32.872270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] PATH_INFO: `/v3/projects/a1b1cdd9480e48a6ab348e45715ef60d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:32.886051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b1cdd9480e48a6ab348e45715ef60d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:32.894057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:32.924081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 893/1788] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:33.150607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b1cdd9480e48a6ab348e45715ef60d', 'request_id': 'req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.163734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:33.165773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:33.185129 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b1cdd9480e48a6ab348e45715ef60d', 'request_id': 'req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.212290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b1cdd9480e48a6ab348e45715ef60d', 'request_id': 'req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.286564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] The token cache is being invalidate because project a1b1cdd9480e48a6ab348e45715ef60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:33.286861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.288877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.300097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 896/1789] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:32 2022] DELETE /identity/v3/projects/a1b1cdd9480e48a6ab348e45715ef60d => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:33.302437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:33.305636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:33.532435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b673e-9b21-43a1-b4af-c7daddfd504b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:33.672476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e61fbc-7e7f-4e5b-8896-0bc9746e1044 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:33.694515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:33.701760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:33.703162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:33.703889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:33.704179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:33.704709 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:33.706746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:33.707174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:33.707529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] PATH_INFO: `/v3/users/604bfa497e654652869c0fcac0064489` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:33.707656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:33.713243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=604bfa497e654652869c0fcac0064489)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:33.726147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:33.739332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:33.760316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:33.778600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b5e3f-0b8d-4815-ac8f-9d9fa8e5cdac service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:33.845871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 894/1790] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:33.933502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604bfa497e654652869c0fcac0064489'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:33.982504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604bfa497e654652869c0fcac0064489'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:34.005364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604bfa497e654652869c0fcac0064489'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:34.019551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604bfa497e654652869c0fcac0064489'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:34.022971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604bfa497e654652869c0fcac0064489'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:34.105651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 897/1791] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:33 2022] DELETE /identity/v3/users/604bfa497e654652869c0fcac0064489 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:35.428401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:35.429210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:35.429317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:35.471466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] MFA Rules not processed for user `6f0d97b196f44ec985db30f86d4ad3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:35.474831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:35.615514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 895/1792] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:35.764620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:35.766376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:35.964400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da33ba-d728-49b8-af09-2b37056b0c90 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:35.974035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:35.975124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:35.976664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:35.977729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] PATH_INFO: `/v3/projects/364b1c4673f4491295b378856bee5343` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:35.981337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364b1c4673f4491295b378856bee5343)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:35.989171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:36.166934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364b1c4673f4491295b378856bee5343', 'request_id': 'req-5d40bc8b-30ff-47a7-a068-3f071affdbd5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.184705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364b1c4673f4491295b378856bee5343', 'request_id': 'req-5d40bc8b-30ff-47a7-a068-3f071affdbd5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.202422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364b1c4673f4491295b378856bee5343', 'request_id': 'req-5d40bc8b-30ff-47a7-a068-3f071affdbd5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.235182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] The token cache is being invalidate because project 364b1c4673f4491295b378856bee5343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:36.236262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.237156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.242682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 898/1793] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:35 2022] DELETE /identity/v3/projects/364b1c4673f4491295b378856bee5343 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:36.243885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:36.257843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708080d-4893-4def-a38f-2575a36df7e0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:36.408014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:36.409809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:36.411893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:36.412352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] PATH_INFO: `/v3/users/d6e6a19f29714ba8984c07758391cf64` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:36.416138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e6a19f29714ba8984c07758391cf64)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:36.419370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:36.454643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:36.456487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:36.487949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40bc8b-30ff-47a7-a068-3f071affdbd5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:36.494926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:36.495654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:36.495944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:36.496215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] PATH_INFO: `/v3/projects/f8e9d80d0e874241ae74b5a97260b97a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:36.497813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e9d80d0e874241ae74b5a97260b97a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:36.500655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:36.549804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:36.576279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e6a19f29714ba8984c07758391cf64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.600562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e6a19f29714ba8984c07758391cf64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.641064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e6a19f29714ba8984c07758391cf64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.641915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e6a19f29714ba8984c07758391cf64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.685039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7f60b8c1-c240-4b2c-868f-b055663feeed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e6a19f29714ba8984c07758391cf64'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.723828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 896/1794] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:36 2022] DELETE /identity/v3/users/d6e6a19f29714ba8984c07758391cf64 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:36.766511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e9d80d0e874241ae74b5a97260b97a', 'request_id': 'req-a21a87ea-7be8-410f-ac34-4494b4609262'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.778749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e9d80d0e874241ae74b5a97260b97a', 'request_id': 'req-a21a87ea-7be8-410f-ac34-4494b4609262'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.793638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e9d80d0e874241ae74b5a97260b97a', 'request_id': 'req-a21a87ea-7be8-410f-ac34-4494b4609262'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.836169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] The token cache is being invalidate because project f8e9d80d0e874241ae74b5a97260b97a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:36.836472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.840777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a21a87ea-7be8-410f-ac34-4494b4609262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:36.846356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 899/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:36 2022] DELETE /identity/v3/projects/f8e9d80d0e874241ae74b5a97260b97a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:36.851480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309865d-047e-40cd-86d6-db57095541ba None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:36.851764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309865d-047e-40cd-86d6-db57095541ba None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:36.852075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309865d-047e-40cd-86d6-db57095541ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:36.899020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f309865d-047e-40cd-86d6-db57095541ba None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:36.900297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f309865d-047e-40cd-86d6-db57095541ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:36.921771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:37.127225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 897/1796] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:37.133825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.134112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.134442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.195753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:37.208970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:37.268969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 900/1797] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:37.287836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309865d-047e-40cd-86d6-db57095541ba None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:37.301633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309865d-047e-40cd-86d6-db57095541ba None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:37.343672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:37.351851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.352146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.352411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.355554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:37.357274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:37.363460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:37.416696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:37.425209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 898/1798] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:37.434598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821cacc-a2ff-46db-83f8-d1734e097e65 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:37.445633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:37.446707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.447220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.447697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.462593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:37.467487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a935e7-d4d6-4652-b7e6-622619cfd76c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:37.561539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 901/1799] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:37 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:37.567527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:37.599935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fddc-248d-44d7-a7b1-d086f582c8b8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:37.670062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d161-66ac-4298-92f4-2cac525a3d4d None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.670346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d161-66ac-4298-92f4-2cac525a3d4d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.670779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d161-66ac-4298-92f4-2cac525a3d4d None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.672515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 902/1800] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:37.684298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.685282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.685665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.731502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:37.733536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:37.810593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:37.811396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.811710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.811992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.813038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:37.814973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:37.879888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 899/1801] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:37 2022] POST /identity/v3/users => generated 284 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:37.880989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:37.911984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2d607-dad5-45df-b2dd-de562ec7459d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:37.926545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:37.927253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:37.927542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:37.927976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:37.928927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:37.939493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:37.968972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 900/1802] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:37.969988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:37.972904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 903/1803] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:34:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:38.009367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a37a5-b129-48d6-895b-73102781ce19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:38.015446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:38.016255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:38.016736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:38.017442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:38.018788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:38.025675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c95d1-9c79-4a73-a902-19ca85b62dda admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:38.039085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 901/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:38.043463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:38.066516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d709448-9eb9-46c0-a491-a03d60be8d1a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:38.073134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:38.074206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:38.074921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:38.075320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] PATH_INFO: `/v3/projects/f840af4aff464ffc8b9da4ea36ced751/users/76cfafd847a142c181cfc93ad664b7df/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:38.118359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f840af4aff464ffc8b9da4ea36ced751, user_id=76cfafd847a142c181cfc93ad664b7df, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:38.124829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:38.154913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 904/1805] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:38 2022] PUT /identity/v3/projects/f840af4aff464ffc8b9da4ea36ced751/users/76cfafd847a142c181cfc93ad664b7df/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:38.170800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:38.171204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:38.171428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:38.262022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] MFA Rules not processed for user `76cfafd847a142c181cfc93ad664b7df`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:38.268755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:38.507243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:38.508016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 902/1806] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:38.513256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:38.597291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:38.602357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:38.680411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:38.782081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b943-f844-4f52-ab05-afb1c23ba3ce admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:38.801204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:38.803617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:38.804207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:38.806080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:38.807148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:38.812447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:38.848124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f1150-d0af-427f-9333-2a25b4f7718e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:38.873755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:38.874416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:38.874752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:38.874979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:38.893346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 905/1807] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:38 2022] POST /identity/v3/projects => generated 369 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:38.893346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:38.895549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:38.899960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:38.901624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:38.944674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 903/1808] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:38.945664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:38.977190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e791e-2d0c-48fc-926f-ef1079136072 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.103022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e847a-27e4-4036-9de0-f5c023ca55ff admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.126701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.127369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.127651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.127926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] PATH_INFO: `/v3/projects/5fd85b240f3346d7b5daf23c1925a298` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.133198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd85b240f3346d7b5daf23c1925a298)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.137908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.160816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.161617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.161864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.162224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.163139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.168997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.219223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 904/1809] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:38 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:39.220915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:39.243180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6417b-6027-4ee8-ab5a-e161d7ba3770 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.259090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.259894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.260183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.260472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.261345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.263016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.290372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 905/1810] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:39.291536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:39.312171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0426ce-4763-484d-b079-5a70c27f02ff admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.332961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd85b240f3346d7b5daf23c1925a298', 'request_id': 'req-14022f22-11b7-47c7-912b-d60d96aae7b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:39.350975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd85b240f3346d7b5daf23c1925a298', 'request_id': 'req-14022f22-11b7-47c7-912b-d60d96aae7b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:39.379638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd85b240f3346d7b5daf23c1925a298', 'request_id': 'req-14022f22-11b7-47c7-912b-d60d96aae7b4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:39.424688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] The token cache is being invalidate because project 5fd85b240f3346d7b5daf23c1925a298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:39.425072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:39.438033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:39.440886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 906/1811] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:38 2022] DELETE /identity/v3/projects/5fd85b240f3346d7b5daf23c1925a298 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:39.444922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:39.449248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:39.483242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.484417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.484948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.485441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] PATH_INFO: `/v3/projects/f55d257d954f4bff99bae91a0ee2b735/users/20cb2bc5a8774df69f3f827ff5778443/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.510193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:39.545587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55d257d954f4bff99bae91a0ee2b735, user_id=20cb2bc5a8774df69f3f827ff5778443, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.550064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.588604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 906/1812] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:39 2022] PUT /identity/v3/projects/f55d257d954f4bff99bae91a0ee2b735/users/20cb2bc5a8774df69f3f827ff5778443/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:39.592859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:39.596932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:39.644699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:39.690097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14022f22-11b7-47c7-912b-d60d96aae7b4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.863106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa80e87-ba94-4f48-a1aa-a44c735c7d8e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.871752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.875956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.877233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.878656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] PATH_INFO: `/v3/users/224694487bfa45eb82ddfa924f439a87` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.882851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.883502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.883793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.884070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:39.884839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=224694487bfa45eb82ddfa924f439a87)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.888238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:39.889965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.906168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:39.935706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 907/1813] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:39.936712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:39.980648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6088cfe-8a84-4c26-bc75-d064f680d82a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:39.998366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:39.999054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:39.999336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:39.999612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] PATH_INFO: `/v3/projects/f55d257d954f4bff99bae91a0ee2b735/users/20cb2bc5a8774df69f3f827ff5778443/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:40.005136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55d257d954f4bff99bae91a0ee2b735, user_id=20cb2bc5a8774df69f3f827ff5778443, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:40.008363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cc59a-8241-4e9d-88b6-7c2ff57e9837 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:40.034213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 908/1814] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:39 2022] PUT /identity/v3/projects/f55d257d954f4bff99bae91a0ee2b735/users/20cb2bc5a8774df69f3f827ff5778443/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:40.052016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:40.056118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:40.056562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:40.078279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224694487bfa45eb82ddfa924f439a87'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:40.109306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] MFA Rules not processed for user `55a5c6359109440fbb66d985fafc24bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:40.110772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:40.140302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224694487bfa45eb82ddfa924f439a87'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:40.176377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224694487bfa45eb82ddfa924f439a87'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:40.177337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224694487bfa45eb82ddfa924f439a87'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:40.179420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-92a5d22e-8ba3-4787-9073-69392494c6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224694487bfa45eb82ddfa924f439a87'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:40.228991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 907/1815] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:39 2022] DELETE /identity/v3/users/224694487bfa45eb82ddfa924f439a87 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:40.230675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:40.234831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:40.235322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:40.272747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 909/1816] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:40.322266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] MFA Rules not processed for user `20cb2bc5a8774df69f3f827ff5778443`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:40.328053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:40.549251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:40.557595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 908/1817] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:40.570994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:40.573590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:40.746271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a073bdd-377a-47ea-9a40-0ee42f1e2f79 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:40.824833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:40.930546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:40.931240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:40.931533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:40.931800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:40.961233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:40.963403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:41.004188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 910/1818] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:41.124787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:41.125622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:41.322167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53335e-950e-4c14-8133-5826b7fbe399 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:41.327728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:41.328680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:41.329523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:41.329523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] PATH_INFO: `/v3/projects/74196d7a38ef438e9a1b1b6bbf5076cd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:41.330902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=74196d7a38ef438e9a1b1b6bbf5076cd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:41.335031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:41.470532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74196d7a38ef438e9a1b1b6bbf5076cd', 'request_id': 'req-4c50573a-c209-4456-996d-926551b4e9da'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.487847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74196d7a38ef438e9a1b1b6bbf5076cd', 'request_id': 'req-4c50573a-c209-4456-996d-926551b4e9da'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.498474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74196d7a38ef438e9a1b1b6bbf5076cd', 'request_id': 'req-4c50573a-c209-4456-996d-926551b4e9da'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.526863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] The token cache is being invalidate because project 74196d7a38ef438e9a1b1b6bbf5076cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:41.527073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.528247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4c50573a-c209-4456-996d-926551b4e9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.531458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 909/1819] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:41 2022] DELETE /identity/v3/projects/74196d7a38ef438e9a1b1b6bbf5076cd => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:41.535708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:41.549456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf88ea6-9ba4-40df-97c8-f7d3d10d1df5 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:41.728125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:41.729722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:41.729722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:41.729722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] PATH_INFO: `/v3/users/729346a7cb634d8789c929c1e470775f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:41.730654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=729346a7cb634d8789c929c1e470775f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:41.736786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:41.901927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729346a7cb634d8789c929c1e470775f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.915826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729346a7cb634d8789c929c1e470775f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.926377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729346a7cb634d8789c929c1e470775f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:41.928010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729346a7cb634d8789c929c1e470775f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:42.009516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ee5bcbb8-a24a-4e71-b77c-0a1cd10ad6d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729346a7cb634d8789c929c1e470775f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:42.046467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 911/1820] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:41 2022] DELETE /identity/v3/users/729346a7cb634d8789c929c1e470775f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:42.669858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:42.670985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:42.672128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:42.674812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 910/1821] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:34:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:43.298205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f03d3-7a66-4b2e-8e90-597eb7f62256 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:43.298785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f03d3-7a66-4b2e-8e90-597eb7f62256 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:43.299099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f03d3-7a66-4b2e-8e90-597eb7f62256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:43.351810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f69f03d3-7a66-4b2e-8e90-597eb7f62256 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:43.354939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f69f03d3-7a66-4b2e-8e90-597eb7f62256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:43.595058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:43.695399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 912/1822] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:34:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:44.173668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:44.178512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.260493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076afc7a-3288-4da9-9c5a-1f452113c913 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:44.265549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:44.266205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.266488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.266779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] PATH_INFO: `/v3/users/76cfafd847a142c181cfc93ad664b7df` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.267944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76cfafd847a142c181cfc93ad664b7df)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:44.269976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:44.317773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:44.337892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76cfafd847a142c181cfc93ad664b7df'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.352991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76cfafd847a142c181cfc93ad664b7df'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.362101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76cfafd847a142c181cfc93ad664b7df'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.363514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76cfafd847a142c181cfc93ad664b7df'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.364278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76cfafd847a142c181cfc93ad664b7df'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.384866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 911/1823] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:44 2022] DELETE /identity/v3/users/76cfafd847a142c181cfc93ad664b7df => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:44.389823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.390432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.390530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.413931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:44.415279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.489684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 913/1824] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:44.500959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:44.502952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.522755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:44.524221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.540044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c76ce6-af2a-4bb6-a9e7-6f37edd4473f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:44.547500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:44.548420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.548763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.549038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] PATH_INFO: `/v3/projects/604fbf0b460144d6b17048ed8bd5877f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.550746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604fbf0b460144d6b17048ed8bd5877f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:44.552724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:44.586596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30be7-d76f-4864-9b86-32c7f4f5701b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:44.590816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:44.592022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.592329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.592766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.604209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:44.605869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e1978-93d4-49a1-834f-08d94974ef44 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:44.616758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604fbf0b460144d6b17048ed8bd5877f', 'request_id': 'req-44bf7031-b95c-469e-9c5e-7c4941a14964'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.621010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 912/1825] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:44.626249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604fbf0b460144d6b17048ed8bd5877f', 'request_id': 'req-44bf7031-b95c-469e-9c5e-7c4941a14964'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.646575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604fbf0b460144d6b17048ed8bd5877f', 'request_id': 'req-44bf7031-b95c-469e-9c5e-7c4941a14964'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.669829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] The token cache is being invalidate because project 604fbf0b460144d6b17048ed8bd5877f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:44.670214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.672695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-44bf7031-b95c-469e-9c5e-7c4941a14964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:44.680613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 914/1826] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:44 2022] DELETE /identity/v3/projects/604fbf0b460144d6b17048ed8bd5877f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:44.683298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.684563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.684948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.720945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:44.722941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.846628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 913/1827] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:44.852342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.852791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.853444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.899885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:44.903871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:44.920570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 915/1828] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:44.920825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:44.934981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288a19-6d6a-40d9-b399-0f41d88a0649 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:44.941229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:44.942032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.942417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.942795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.943652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:44.944066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:44.945813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:44.962920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 914/1829] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:44.966845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:44.977228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e6259-1cee-449b-842b-0c57cd88bda4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:44.981682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:44.982298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:44.982574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:44.982860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:44.983483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:44.985417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.022845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 916/1830] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:44 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:45.026958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.039070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43875c3-b280-4846-a6c8-71418de4d7f0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.080125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:45.114333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.114971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.115229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.115501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.116366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.118367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.143615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 915/1831] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:45.149493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.161882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a350e1-7b77-40c3-aaad-345f66a9c39e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.166996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.167709 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.168057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.168417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.169648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.171115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.181471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 917/1832] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:45.185372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.196128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20720084-a75e-4bfb-9238-62293f9ab684 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.200974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.201578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.201858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.202131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.202777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.204391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefa74b-c416-4b90-9b1e-568d4322e363 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.215429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 916/1833] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:45.219785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.231095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362ecb4-d414-4eb4-94ea-2f72f2b241e4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.238444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.239472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.240103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.240770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] PATH_INFO: `/v3/projects/296bddca93f041edb07f4f631deea1d9/users/5a606405a34c46afb3dd654da6d7bb95/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.256980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] RBAC: Authorizing `identity:create_grant(project_id=296bddca93f041edb07f4f631deea1d9, user_id=5a606405a34c46afb3dd654da6d7bb95, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.259618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.276153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 918/1834] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:45 2022] PUT /identity/v3/projects/296bddca93f041edb07f4f631deea1d9/users/5a606405a34c46afb3dd654da6d7bb95/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:45.287170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.287716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.288009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.320641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] MFA Rules not processed for user `5a606405a34c46afb3dd654da6d7bb95`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:45.322006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.400899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.403008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.408202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:45.420339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 917/1835] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:45.425014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.523466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b2cad-8fae-4191-b523-b36dcb69603d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.535708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.537103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.538805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.539877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] PATH_INFO: `/v3/projects/4286889c73eb4a848c46f81449e062d6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.543386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4286889c73eb4a848c46f81449e062d6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.546128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.555964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38871cd-a024-43ef-ae70-1699a841fa45 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.568726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.569442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.569878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.570159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.571241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.574846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.617510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 918/1836] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/projects => generated 391 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:45.621669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.629757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4286889c73eb4a848c46f81449e062d6', 'request_id': 'req-0159cf98-63a6-40a0-9320-de0ec4a94387'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:45.636448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4286889c73eb4a848c46f81449e062d6', 'request_id': 'req-0159cf98-63a6-40a0-9320-de0ec4a94387'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:45.648232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4286889c73eb4a848c46f81449e062d6', 'request_id': 'req-0159cf98-63a6-40a0-9320-de0ec4a94387'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:45.668099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:45.670264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] The token cache is being invalidate because project 4286889c73eb4a848c46f81449e062d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:45.671581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:45.674030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0159cf98-63a6-40a0-9320-de0ec4a94387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:45.677308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 919/1837] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:45 2022] DELETE /identity/v3/projects/4286889c73eb4a848c46f81449e062d6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:45.684551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fadc3-324f-4cc7-8be9-ddf5fe3d3162 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.684776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fadc3-324f-4cc7-8be9-ddf5fe3d3162 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.685232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fadc3-324f-4cc7-8be9-ddf5fe3d3162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.714519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f75fadc3-324f-4cc7-8be9-ddf5fe3d3162 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:45.716067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f75fadc3-324f-4cc7-8be9-ddf5fe3d3162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.734128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f4e26-402f-478c-a141-9342ce073869 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.742325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.745334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.746138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.746618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.747768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.750161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.789450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 919/1838] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/users => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:45.794031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.796335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.813152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 920/1839] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:45.814320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fe016-0d20-444c-839d-06a6d3b0915c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.819816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.820445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.820758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.821019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.821774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.821874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.822130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.822412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.823411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.840231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 920/1840] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:45.841262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.846583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:45.848696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.854773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8feaa2-a760-4965-88fe-a5e19c753140 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.860716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 921/1841] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:45.861100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.861926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.862256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.862632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] PATH_INFO: `/v3/projects/0a4151ac82cc409e8fd7c9b96f628903/users/871fde62013e4896b23f369fe3b6de14/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.864194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.876908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1d468-4074-4905-98d1-aef62bc22c17 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.881504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4151ac82cc409e8fd7c9b96f628903, user_id=871fde62013e4896b23f369fe3b6de14, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.881986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:45.882915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:45.883873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:45.884229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:45.885008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.885130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:45.885393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:45.887127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:45.904316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 922/1842] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:45.908492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.908730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 921/1843] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:45 2022] PUT /identity/v3/projects/0a4151ac82cc409e8fd7c9b96f628903/users/871fde62013e4896b23f369fe3b6de14/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:45.910979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:45.913239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:45.926512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494c3a-f50a-4fd8-b321-e1f976faf15d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:45.937828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:46.014229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.015045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.015331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.015685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] PATH_INFO: `/v3/projects/f840af4aff464ffc8b9da4ea36ced751` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.016990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f840af4aff464ffc8b9da4ea36ced751)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.017768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508c21e-b79e-4330-b0e4-b66611d17918 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.020403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.024400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.031325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.032736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.034678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.035597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.038664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.078709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 922/1844] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:45 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:46.082637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.094796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f840af4aff464ffc8b9da4ea36ced751', 'request_id': 'req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.101812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f840af4aff464ffc8b9da4ea36ced751', 'request_id': 'req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.122902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f840af4aff464ffc8b9da4ea36ced751', 'request_id': 'req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.142035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] The token cache is being invalidate because project f840af4aff464ffc8b9da4ea36ced751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:46.142638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.143201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.145519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 923/1845] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:45 2022] DELETE /identity/v3/projects/f840af4aff464ffc8b9da4ea36ced751 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:46.146349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.147737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.175212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac7ab8-094a-4d3e-8d16-5c93ef6953b3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.181219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.181930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.182270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.182606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.183317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.185748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.204354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 923/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:46.205691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.208871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.220543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:46.252501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8097d-bcfa-434a-b9ec-a5a7a67a827c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.259273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.259764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.260044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.260317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.260998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.262952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.291971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 924/1847] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:46 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:46.294347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.296074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.311179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bd717-fd59-4760-8e4a-a72a715097a9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.319869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.322081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.322534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.322997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] PATH_INFO: `/v3/users/20cb2bc5a8774df69f3f827ff5778443` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.324337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20cb2bc5a8774df69f3f827ff5778443)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.327108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.335016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78815f33-5273-43fe-a35c-933fa7bdb84a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.354117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.354117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.354117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.354420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.356102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.369288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.370879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.376479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:46.383052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 924/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:46.383763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.385957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.407447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20cb2bc5a8774df69f3f827ff5778443'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.408188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8ee17-1794-460c-9fd6-e8698833535d service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.415507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.417090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.417246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.417393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] PATH_INFO: `/v3/projects/0a4151ac82cc409e8fd7c9b96f628903/users/871fde62013e4896b23f369fe3b6de14/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.419902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4151ac82cc409e8fd7c9b96f628903, user_id=871fde62013e4896b23f369fe3b6de14, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.422579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20cb2bc5a8774df69f3f827ff5778443'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.422764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.442588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 925/1849] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:46 2022] PUT /identity/v3/projects/0a4151ac82cc409e8fd7c9b96f628903/users/871fde62013e4896b23f369fe3b6de14/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:46.443606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.451626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20cb2bc5a8774df69f3f827ff5778443'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.452776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20cb2bc5a8774df69f3f827ff5778443'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.478549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5e04808e-4754-4020-9f39-23e5efd77e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20cb2bc5a8774df69f3f827ff5778443'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:46.488420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 925/1850] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:46 2022] DELETE /identity/v3/users/20cb2bc5a8774df69f3f827ff5778443 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:46.492743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.493030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.493337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.548229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] MFA Rules not processed for user `871fde62013e4896b23f369fe3b6de14`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:46.549555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.587400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998dff0-5276-4026-8183-2671dd1e42db admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.738948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.740704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 926/1851] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:46.741698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.743155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.744122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.745269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.749744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.754330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.758191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.786663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 926/1852] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:46.791017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.793241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:46.849222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa32c7-e9c7-43a6-ab1e-9afec56dc518 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.866865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.867921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.868439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.869840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.871693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.873941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.908188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 927/1853] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:46.910116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:46.922222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf974-f1c1-443e-ad23-7fdf03c936b6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.935131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.937153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.938574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.939002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] PATH_INFO: `/v3/projects/3142556cdb7e4621a020420f973c550d/users/1f1c2cc044ea419b850bff0c2dbb69ef/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.942707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63e8de-a48d-4388-9b4d-f2c7fee11679 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:46.953944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:46.955896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:46.956683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:46.957071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:46.957710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:46.966055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3142556cdb7e4621a020420f973c550d, user_id=1f1c2cc044ea419b850bff0c2dbb69ef, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.969992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884e163-d975-4fae-816a-75c29459e929 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:46.984098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:46.989655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.004560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 928/1854] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:46 2022] PUT /identity/v3/projects/3142556cdb7e4621a020420f973c550d/users/1f1c2cc044ea419b850bff0c2dbb69ef/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:47.006829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.007761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.008702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.025706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 927/1855] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:47.051598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] MFA Rules not processed for user `1f1c2cc044ea419b850bff0c2dbb69ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:47.052858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:47.113536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.116277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:47.133931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7827aa9-d36e-46b4-9728-bda2ed92f122 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.164042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 929/1856] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:47.164426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.199393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:47.239428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.240361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.241269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.241761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.244288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:47.292490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506821e-b53f-49e2-aa0d-03f8ae8517fd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.302642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.306665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.306665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.306665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.306665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.307707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.358217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 930/1857] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:47 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:47.359630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.460309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.464633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.495507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 928/1858] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:47.527835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b51ef4-a04f-4210-82a5-2f717621bcbf admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.535865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.536898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.537375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.537868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.538801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.541988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.553945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:47.587039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 931/1859] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:47 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:47.591055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.621760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bea982-db0d-4a57-9bde-51230f3e4b56 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.630474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.632262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.633142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.633346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.634642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.639073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.666856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 929/1860] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:47.667150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.690303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ebe7-de29-485c-b081-2364fe06969f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.699289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.700449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.701051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.701533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] PATH_INFO: `/v3/projects/2fe217f99e1049158e9226bfb7d685ed/users/aabe9a52304649309908e728c7a847cf/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.735004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe217f99e1049158e9226bfb7d685ed, user_id=aabe9a52304649309908e728c7a847cf, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.741291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.785304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 932/1861] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:47 2022] PUT /identity/v3/projects/2fe217f99e1049158e9226bfb7d685ed/users/aabe9a52304649309908e728c7a847cf/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:47.790347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.808659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b538b-1c3f-4cb0-b9bd-4b0edb0eb583 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.919037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.919923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.920297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.920743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.921815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:47.923980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:47.937893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.939735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:47.962931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 930/1862] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:47.965682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:47.978414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffdf30-8164-4cad-9b9e-b1372c7bf2b1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.990320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:47.992057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7176a8-6d84-4908-bd19-0a1f26ba11ee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:47.993155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:47.993532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:47.994104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] PATH_INFO: `/v3/projects/f55d257d954f4bff99bae91a0ee2b735` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:47.997909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f55d257d954f4bff99bae91a0ee2b735)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.004119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.005202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.007096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.009184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.009184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] PATH_INFO: `/v3/projects/2fe217f99e1049158e9226bfb7d685ed/users/aabe9a52304649309908e728c7a847cf/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.013624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe217f99e1049158e9226bfb7d685ed, user_id=aabe9a52304649309908e728c7a847cf, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.031519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f55a9-6edf-4963-825a-9d33fbe72a40 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.085285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 931/1863] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:47 2022] PUT /identity/v3/projects/2fe217f99e1049158e9226bfb7d685ed/users/aabe9a52304649309908e728c7a847cf/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:48.091048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f55d257d954f4bff99bae91a0ee2b735', 'request_id': 'req-3aa7d457-93be-447e-b802-dde2b97ce0c5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:48.096515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.099720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.100011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.103628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f55d257d954f4bff99bae91a0ee2b735', 'request_id': 'req-3aa7d457-93be-447e-b802-dde2b97ce0c5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:48.118504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f55d257d954f4bff99bae91a0ee2b735', 'request_id': 'req-3aa7d457-93be-447e-b802-dde2b97ce0c5'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:48.167645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] The token cache is being invalidate because project f55d257d954f4bff99bae91a0ee2b735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:48.168027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:48.170330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3aa7d457-93be-447e-b802-dde2b97ce0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:48.175119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 933/1864] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:47 2022] DELETE /identity/v3/projects/f55d257d954f4bff99bae91a0ee2b735 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:48.180702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eb719-e535-4149-b9e9-4c88ce87a352 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.181702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eb719-e535-4149-b9e9-4c88ce87a352 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.182166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754eb719-e535-4149-b9e9-4c88ce87a352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.250832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-754eb719-e535-4149-b9e9-4c88ce87a352 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:48.252252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-754eb719-e535-4149-b9e9-4c88ce87a352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:48.280221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] MFA Rules not processed for user `aabe9a52304649309908e728c7a847cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:48.281513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:48.405745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 934/1865] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:48.413327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.413624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.413951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.446513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:48.455491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:48.458333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:48.480894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 935/1866] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:48.482640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 932/1867] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:48.487011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.499736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.501415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:48.510272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927b0a3-0aad-434a-a214-653ef460a30d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.516265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.516963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.517435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.517655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.519545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:48.520230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.521987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.540301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 936/1868] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:48.544536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.556304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7c11d-7b5c-4376-b956-51e3110483f6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.561669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.562284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.562748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.563033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.563673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.568495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.623371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 937/1869] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/projects => generated 399 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:48.624486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.648002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:48.655533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71101755-ff8a-43b5-bdce-1498a394e5d7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.739555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.740181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.740462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.740765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.752138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab00fed-34bc-4057-8cf8-69cae9e204ee admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.762418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.763243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.763527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.763801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.764446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.766453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.804988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 938/1870] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:48.805778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.818125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266885e-dc52-46d4-96e4-c557f01f611f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.824275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.825018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.825383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.825720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.826490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.828267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.839474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.841362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.844768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 939/1871] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:48.845716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.852371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 933/1872] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:48.856973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b2cc-d0cb-4f7e-95ce-38a7c52f1097 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.862155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.863785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.863863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.864339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.865087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.867648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30ec1a-fbff-4c08-add3-24f16eb34dd5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.880259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 940/1873] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:48.884116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:48.905867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1e11f-beb4-4a05-9c6b-41e977b78182 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:48.937439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:48.938137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.938417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.938691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] PATH_INFO: `/v3/projects/f2c7c1e265a143ef8244135b80c68c0c/users/f0d094ddadb94ceabef332c01106ea7c/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:48.959619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c7c1e265a143ef8244135b80c68c0c, user_id=f0d094ddadb94ceabef332c01106ea7c, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:48.962798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:48.984631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 934/1874] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:48 2022] PUT /identity/v3/projects/f2c7c1e265a143ef8244135b80c68c0c/users/f0d094ddadb94ceabef332c01106ea7c/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:48.995413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905febea-e385-422b-a628-2a07d365b0cb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:48.995638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905febea-e385-422b-a628-2a07d365b0cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:48.995973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905febea-e385-422b-a628-2a07d365b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:49.046566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-905febea-e385-422b-a628-2a07d365b0cb None None] MFA Rules not processed for user `f0d094ddadb94ceabef332c01106ea7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:49.047986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-905febea-e385-422b-a628-2a07d365b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:49.343485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 941/1875] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:49.348407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:49.424240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:49.556743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377317-471d-45c8-90df-544358524c44 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:49.571054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:49.571878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:49.572279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:49.572674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:49.574525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:49.579725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:49.642529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 935/1876] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:49 2022] POST /identity/v3/projects => generated 399 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:49.647458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905febea-e385-422b-a628-2a07d365b0cb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:49.664810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905febea-e385-422b-a628-2a07d365b0cb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:49.781149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:49.782778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:49.785123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:49.785850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:49.790730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:49.796932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:49.839981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:49.871667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 942/1877] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:49 2022] POST /identity/v3/users => generated 306 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:49.876745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:49.906948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a568d9-9eb1-44f8-b50d-fc3c9b53618d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:49.914827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:49.916388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:49.917077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:49.917509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:49.918337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:49.921572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:49.938809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 936/1878] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:49.944015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:49.965734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574bbc8-6fe1-4ccc-8392-e0fd25c896de admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:49.979332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:49.981582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:49.981983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:49.982446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] PATH_INFO: `/v3/projects/a7c2f09e8d544951bce8f9747db12c47/users/19b9ddc7f27c4938b7d5e1251da2e73d/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.018509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c2f09e8d544951bce8f9747db12c47, user_id=19b9ddc7f27c4938b7d5e1251da2e73d, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:50.024455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:50.055685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:50.057865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 943/1879] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:49 2022] PUT /identity/v3/projects/a7c2f09e8d544951bce8f9747db12c47/users/19b9ddc7f27c4938b7d5e1251da2e73d/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:50.072370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5939-05c0-4280-8b7e-8c4724c0ccea admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:50.193907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:50.196883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.258862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:50.260205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.260731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.261160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.262250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:50.266267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:50.285443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 937/1880] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:50.290304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:50.318033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36541721-2fb4-40d9-b3c1-949e274e010c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:50.326930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9539b-97af-419a-9d32-77788b74d57d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:50.328566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:50.329898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.330745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.331520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] PATH_INFO: `/v3/projects/a7c2f09e8d544951bce8f9747db12c47/users/19b9ddc7f27c4938b7d5e1251da2e73d/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.336281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c2f09e8d544951bce8f9747db12c47, user_id=19b9ddc7f27c4938b7d5e1251da2e73d, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:50.337033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:50.349577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b257a9-fb51-4c31-846d-f12679ba7696 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:50.356670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.372410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.372939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] PATH_INFO: `/v3/users/1f1c2cc044ea419b850bff0c2dbb69ef` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.375769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1c2cc044ea419b850bff0c2dbb69ef)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:50.379729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:50.381180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 938/1881] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:50 2022] PUT /identity/v3/projects/a7c2f09e8d544951bce8f9747db12c47/users/19b9ddc7f27c4938b7d5e1251da2e73d/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:50.392684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.393148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.393602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.454664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] MFA Rules not processed for user `19b9ddc7f27c4938b7d5e1251da2e73d`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:50.456616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.498037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1c2cc044ea419b850bff0c2dbb69ef'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:50.518917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1c2cc044ea419b850bff0c2dbb69ef'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:50.528571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1c2cc044ea419b850bff0c2dbb69ef'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:50.529629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1c2cc044ea419b850bff0c2dbb69ef'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:50.531647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e66c90f9-cc21-4ebb-8698-6a05bba97d33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1c2cc044ea419b850bff0c2dbb69ef'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:50.564951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 944/1882] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:50 2022] DELETE /identity/v3/users/1f1c2cc044ea419b850bff0c2dbb69ef => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:50.569756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.570042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.570351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.621091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:50.627139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.648107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 939/1883] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:50 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:50.654104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:50.657199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.681928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:50.739002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:50.785972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 945/1884] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:50.798786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc0ab4-d199-4fc4-91f5-c258769d225a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:50.855918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:50.858181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.950729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:50.951973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:50.952467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:50.952977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:50.996388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1128fd-4f7f-4c0f-b394-c89ea82ce12b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:51.002914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:51.004267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:51.004764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:51.005109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:51.021796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:51.023214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:51.052730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 946/1885] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:51.115752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:51.118820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd2375-da48-4859-a15c-efddbbd0e42d service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:51.171795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 940/1886] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:50 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:51.299027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:51.302724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:51.325041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc309e-e39d-4c9c-a607-cd87e2a74c24 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:51.330581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:51.331357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:51.331655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:51.331905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:51.334007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:51.449225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:51.451916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c073cd-66fc-403e-bb05-1ee66e8363f6 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:51.483130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 947/1887] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:52.332925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:52.333504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:52.334954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:52.340681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 941/1888] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:34:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:34:52.351677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:52.352639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:52.353503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:52.521007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:52.523433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:52.651039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:52.653797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:52.702524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654a080-ee98-4f37-b58f-f9f52a936f84 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:52.731746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:52.732431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:52.732755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:52.733038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] PATH_INFO: `/v3/projects/3142556cdb7e4621a020420f973c550d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:52.737461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3142556cdb7e4621a020420f973c550d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:52.741474 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:52.854800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 948/1889] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:52.944501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3142556cdb7e4621a020420f973c550d', 'request_id': 'req-df497392-fcde-4673-a496-a89d70cf7777'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:52.967200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3142556cdb7e4621a020420f973c550d', 'request_id': 'req-df497392-fcde-4673-a496-a89d70cf7777'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:52.997799 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3142556cdb7e4621a020420f973c550d', 'request_id': 'req-df497392-fcde-4673-a496-a89d70cf7777'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:52.998168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:53.000691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.022481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56578d-3b56-42cd-b0a6-6125ee14d657 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:53.035401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:53.049374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:53.058676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] The token cache is being invalidate because project 3142556cdb7e4621a020420f973c550d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:53.058995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.060124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.063867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 942/1890] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:52 2022] DELETE /identity/v3/projects/3142556cdb7e4621a020420f973c550d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:53.066883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:53.069171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.109700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:53.110862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.111343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.111806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] PATH_INFO: `/v3/users/5a606405a34c46afb3dd654da6d7bb95` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.114047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a606405a34c46afb3dd654da6d7bb95)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:53.116563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:53.218831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a606405a34c46afb3dd654da6d7bb95'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.265783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a606405a34c46afb3dd654da6d7bb95'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.284180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df497392-fcde-4673-a496-a89d70cf7777 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:53.285813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a606405a34c46afb3dd654da6d7bb95'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.291180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a606405a34c46afb3dd654da6d7bb95'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.292761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a606405a34c46afb3dd654da6d7bb95'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.294103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:53.295345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.295897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.296616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.300199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.325496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 949/1891] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:52 2022] DELETE /identity/v3/users/5a606405a34c46afb3dd654da6d7bb95 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:53.325496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:53.325780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:53.325780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5d1b6-c751-4596-991f-363a52b1b85b service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:53.325950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.340217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 943/1892] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:53.345661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.345661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.345661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.394276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:53.396513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.477719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4ee5-9104-4246-8249-23c1ac7bbd4a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:53.490684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:53.491496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.491886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.492287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] PATH_INFO: `/v3/users/aabe9a52304649309908e728c7a847cf` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.495556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabe9a52304649309908e728c7a847cf)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:53.497704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:53.554942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 944/1893] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:53.566042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:53.567727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.604460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:53.609377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabe9a52304649309908e728c7a847cf'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.665854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabe9a52304649309908e728c7a847cf'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.679758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabe9a52304649309908e728c7a847cf'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.681180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabe9a52304649309908e728c7a847cf'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.682323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabe9a52304649309908e728c7a847cf'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:53.708995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71960-5696-4406-aede-3c29f32cf903 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:53.712061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 950/1894] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:53 2022] DELETE /identity/v3/users/aabe9a52304649309908e728c7a847cf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:53.714809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:53.715381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.715789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.716076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.717873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.733334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:53.734939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:53.755344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 945/1895] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:54.368727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.370908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.376804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.380063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.408628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48652f35-6afa-4363-a409-60d307a3e2a1 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.499219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:54.533611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc549f61-3c69-4958-9040-b9b541fda50c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.539529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.540599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.541982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.542520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] PATH_INFO: `/v3/projects/296bddca93f041edb07f4f631deea1d9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.546868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] RBAC: Authorizing `identity:delete_project(project_id=296bddca93f041edb07f4f631deea1d9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.550271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.571213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.572371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.572923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.573411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] PATH_INFO: `/v3/projects/2fe217f99e1049158e9226bfb7d685ed` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.575020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe217f99e1049158e9226bfb7d685ed)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.577268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.645379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296bddca93f041edb07f4f631deea1d9', 'request_id': 'req-1fc4a33e-16d3-46c6-9753-2c8e2682719b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.648448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe217f99e1049158e9226bfb7d685ed', 'request_id': 'req-b3140360-9dde-4f98-94ad-87b0f709e0b3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.657200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296bddca93f041edb07f4f631deea1d9', 'request_id': 'req-1fc4a33e-16d3-46c6-9753-2c8e2682719b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.659807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe217f99e1049158e9226bfb7d685ed', 'request_id': 'req-b3140360-9dde-4f98-94ad-87b0f709e0b3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.667535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296bddca93f041edb07f4f631deea1d9', 'request_id': 'req-1fc4a33e-16d3-46c6-9753-2c8e2682719b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.673151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe217f99e1049158e9226bfb7d685ed', 'request_id': 'req-b3140360-9dde-4f98-94ad-87b0f709e0b3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.685461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] The token cache is being invalidate because project 296bddca93f041edb07f4f631deea1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:54.685916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.686700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.693188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 951/1896] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:54 2022] DELETE /identity/v3/projects/296bddca93f041edb07f4f631deea1d9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:54.694728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.694837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] The token cache is being invalidate because project 2fe217f99e1049158e9226bfb7d685ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:54.695119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.696925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-b3140360-9dde-4f98-94ad-87b0f709e0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.698956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.728707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 946/1897] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:54 2022] DELETE /identity/v3/projects/2fe217f99e1049158e9226bfb7d685ed => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:54.731155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3acd-c785-4119-aaee-a10acb7e5b87 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.731450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3acd-c785-4119-aaee-a10acb7e5b87 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.731787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e3acd-c785-4119-aaee-a10acb7e5b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.764415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-df4e3acd-c785-4119-aaee-a10acb7e5b87 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:54.765777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-df4e3acd-c785-4119-aaee-a10acb7e5b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.780401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:54.812572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4a33e-16d3-46c6-9753-2c8e2682719b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.874429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 947/1898] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:54.883167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.883550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.883898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.896138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.896138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.896138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.896892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] PATH_INFO: `/v3/users/871fde62013e4896b23f369fe3b6de14` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.903173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=871fde62013e4896b23f369fe3b6de14)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.909666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.917239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:54.918552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.927908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 948/1899] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:54.938437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.948397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23127e1-45b8-45a3-9c72-5a0eefd72bb0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.961508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.962762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.963491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.965194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.966205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:54.966753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.969470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.997625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 949/1900] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:54.999154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.017258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f626-2e30-4204-aaa9-848aaf9d2d4d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.023341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '871fde62013e4896b23f369fe3b6de14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.030848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.031884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.032251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.032533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.033540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.035664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.047790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '871fde62013e4896b23f369fe3b6de14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.063527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '871fde62013e4896b23f369fe3b6de14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.064361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '871fde62013e4896b23f369fe3b6de14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.065161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '871fde62013e4896b23f369fe3b6de14'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.080032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 950/1901] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:54 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:55.085025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.086865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.087371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 952/1902] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:54 2022] DELETE /identity/v3/users/871fde62013e4896b23f369fe3b6de14 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:55.111504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:55.181093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf67b9-4525-47ad-a3a5-910e369123f7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.267075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.267307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.267515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.267962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.268432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.271197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.302735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 951/1903] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:55.308002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.321683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1980f-1ca5-4e0e-a5f9-0790ce68a648 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.326728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.327412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.327679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.328429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.328681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.330357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.343018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 953/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:55.347218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.358272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ce1b4-2f99-4f9b-b4cd-9c1a7788b872 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.364637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.365436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.365925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.366423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.367183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.368942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.384635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 952/1905] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:55.386644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.398223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc1fe0-018d-4fe2-b957-764702782469 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.404253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.405172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.405711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.406141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] PATH_INFO: `/v3/projects/2d4a4d67c85740d09e766ad2fc14952a/users/ed5a6f71105d4201afd740ae6f428cba/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.431649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4a4d67c85740d09e766ad2fc14952a, user_id=ed5a6f71105d4201afd740ae6f428cba, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.434754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef0f6-9e98-4444-9626-9b4fffa1777a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.456984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 954/1906] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:55 2022] PUT /identity/v3/projects/2d4a4d67c85740d09e766ad2fc14952a/users/ed5a6f71105d4201afd740ae6f428cba/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:55.465707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.467761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.470650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.472857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.473410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.481372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228ac4c-70a7-49f8-b1b5-372c0640cd9e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.517558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] MFA Rules not processed for user `ed5a6f71105d4201afd740ae6f428cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.518831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.551824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.552792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.553609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.554058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] PATH_INFO: `/v3/projects/0a4151ac82cc409e8fd7c9b96f628903` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.555633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4151ac82cc409e8fd7c9b96f628903)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.560435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:55.564151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.629695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:55.635194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4151ac82cc409e8fd7c9b96f628903', 'request_id': 'req-2adb760d-b97d-4fe2-b44c-a6178f766950'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.644334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4151ac82cc409e8fd7c9b96f628903', 'request_id': 'req-2adb760d-b97d-4fe2-b44c-a6178f766950'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.646327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 955/1907] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:55.651516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.661235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4151ac82cc409e8fd7c9b96f628903', 'request_id': 'req-2adb760d-b97d-4fe2-b44c-a6178f766950'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.676933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] The token cache is being invalidate because project 0a4151ac82cc409e8fd7c9b96f628903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:55.677383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.678163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2adb760d-b97d-4fe2-b44c-a6178f766950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.679575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ab6ec-0e2f-4b98-8a8f-cc1b9b594437 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.683044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 953/1908] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:55 2022] DELETE /identity/v3/projects/0a4151ac82cc409e8fd7c9b96f628903 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:55.684931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942283-3427-4fc7-984e-7f2e73f473ff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.685316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942283-3427-4fc7-984e-7f2e73f473ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.685686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90942283-3427-4fc7-984e-7f2e73f473ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.731301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-90942283-3427-4fc7-984e-7f2e73f473ff None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.734006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90942283-3427-4fc7-984e-7f2e73f473ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.798097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.799069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.799356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.799813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.800631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.803661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.839881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 954/1909] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:55.849229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.849624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.849977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.855312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 956/1910] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:55.864286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.881922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.884184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.887441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.979460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b366e8-6ef9-49af-a038-8d44f281f2d2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.986255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.987133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.988008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.988285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.988965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.991989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.004929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 955/1911] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:56.009381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.021020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141b0e5-d5f3-43d7-b800-97ab15baeb39 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.024466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 957/1912] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/users => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:56.028116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.029953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.030783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.031215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.031637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.032447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:56.032925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.035652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.044121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b8454-f433-4b79-837e-14ef42225885 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.051475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.051954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 956/1913] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:56.052229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.052637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.053017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.054269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.055834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.056187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.074576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 958/1914] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.074766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0dd447-c427-4050-a1d6-a10c5bba9007 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.075952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.079602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.080475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.080795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.081608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.082253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.084175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.092755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ee03-39de-4a43-9241-f90c0baa5c80 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.098713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.100194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.100819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.101287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] PATH_INFO: `/v3/projects/867d22d866ef4f38a5101b4e1f7b3010/users/a0887eb44270424bb6ed65078b454006/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.125747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 957/1915] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:56.126898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.135962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=867d22d866ef4f38a5101b4e1f7b3010, user_id=a0887eb44270424bb6ed65078b454006, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.143070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.187984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 959/1916] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:56 2022] PUT /identity/v3/projects/867d22d866ef4f38a5101b4e1f7b3010/users/a0887eb44270424bb6ed65078b454006/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.189290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.222922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:56.279701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:56.338267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fe7e8-4215-4ce1-a0d9-17912cffcaf3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.374340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da016d4-8f4f-453b-bd5c-b0617b34d068 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.382246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.383090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.383646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.384030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.385413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.390112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.419052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 960/1917] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.421253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.428220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.449027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.449655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.449935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.450208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.451000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.451482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b906e-2d9d-4cfb-8b28-498a9e52be12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.453181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.457956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.458814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.459205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.459582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] PATH_INFO: `/v3/users/f0d094ddadb94ceabef332c01106ea7c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.460972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d094ddadb94ceabef332c01106ea7c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.463347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.480785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 958/1918] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:56.482326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.499476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d62505-ac83-41bb-90b9-5582fd5d5cc9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.507761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.508448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.508761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.509036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] PATH_INFO: `/v3/projects/867d22d866ef4f38a5101b4e1f7b3010/users/a0887eb44270424bb6ed65078b454006/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.511275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=867d22d866ef4f38a5101b4e1f7b3010, user_id=a0887eb44270424bb6ed65078b454006, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.514004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.530745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 959/1919] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:56 2022] PUT /identity/v3/projects/867d22d866ef4f38a5101b4e1f7b3010/users/a0887eb44270424bb6ed65078b454006/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.532193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.560055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d094ddadb94ceabef332c01106ea7c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:56.598364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d094ddadb94ceabef332c01106ea7c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:56.607952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d094ddadb94ceabef332c01106ea7c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:56.609114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d094ddadb94ceabef332c01106ea7c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:56.609845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d094ddadb94ceabef332c01106ea7c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:56.613909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46576-3a09-446b-82fd-7f7a4b77b30a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.620632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.620632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.620632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.621002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.621598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.624901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0d34-fbdd-44ed-bba0-e01738912484 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.638497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 960/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.638497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.638497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.638497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.640780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 961/1921] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:56 2022] DELETE /identity/v3/users/f0d094ddadb94ceabef332c01106ea7c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.641836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.643237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.669264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] MFA Rules not processed for user `a0887eb44270424bb6ed65078b454006`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:56.670557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.723433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ac1d3-989e-461f-b40a-2312e8d14b08 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.732266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:56.739118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.739757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.740337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.740652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.741422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.742965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8552f-dcb9-4d38-aa6f-a6ae7de0799d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.761567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 962/1922] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.762869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.763166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.763902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.766200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 961/1923] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:56.767342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.779974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9d9ac-919c-44b1-9b73-f364774d7c53 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.789343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:56.790585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.791013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.791502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.791874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.792300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] PATH_INFO: `/v3/projects/0448a001b4714952b7d9767ec2d2cacc/users/379dcb04a9e046259508a83ef1424274/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.799720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 963/1924] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:56.800665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.801947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.831608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:56.833177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0448a001b4714952b7d9767ec2d2cacc, user_id=379dcb04a9e046259508a83ef1424274, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.837098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.868783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 962/1925] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:56 2022] PUT /identity/v3/projects/0448a001b4714952b7d9767ec2d2cacc/users/379dcb04a9e046259508a83ef1424274/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.870649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.876634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.923971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a81bd-7f10-48e4-94e8-8179b07303c2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.981227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427d611-b759-4af5-873c-75b865ea6fdd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.986543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.987208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.987626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.988026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.993140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.994208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.995258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.995546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.079465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.081017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1adcb-80e8-4aac-94a0-e3bdb7914d62 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.092457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.093989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 964/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:57.094502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.107722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.108016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.108296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.119348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 963/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:57.177224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] MFA Rules not processed for user `379dcb04a9e046259508a83ef1424274`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:57.180427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:57.253810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.264252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4c18-13bb-4d9c-89b8-afd7d99c8018 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.270040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.270679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.270955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.271238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.273955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:57.318695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 965/1928] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:57.320397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.333998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb91e26-6928-4bcb-bac8-3c2b6fc9cc38 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.339411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.340741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.341030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.341408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.342616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.345013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.352868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:57.369805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.371697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.384428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 964/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:57.398683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 966/1930] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:57 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:57.403419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.416638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4d61c-67d2-4df1-8e9f-5de4bd3a633b service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.421225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:57.559548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.559548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.559548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.559548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.561714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.562401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.605147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 965/1931] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:57 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:57.608697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.633247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.633839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:57.637177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8eee7-ace9-49e9-a18f-85d42501c558 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.651899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.652527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.654447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.654447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.654855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.658683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e514c-0d07-4e9d-ad2c-7bceb0734807 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.658912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.670244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.671024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.671170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.671438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] PATH_INFO: `/v3/projects/f2c7c1e265a143ef8244135b80c68c0c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.674351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c7c1e265a143ef8244135b80c68c0c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.675610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.685998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 967/1932] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:57.686980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.697791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5adc-e349-4b55-9aab-3ccddcb3efb8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.703011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.703700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.703984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.704260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] PATH_INFO: `/v3/projects/1dc2295f232841798892910f44d3281c/users/83b277de18514ab7ab0dee616ac1df5b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.735199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc2295f232841798892910f44d3281c, user_id=83b277de18514ab7ab0dee616ac1df5b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.755108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.791517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 968/1933] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:57 2022] PUT /identity/v3/projects/1dc2295f232841798892910f44d3281c/users/83b277de18514ab7ab0dee616ac1df5b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:57.795750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.833869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c7c1e265a143ef8244135b80c68c0c', 'request_id': 'req-810443d6-1e94-483b-99b6-2ee77282ad55'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:57.848122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c7c1e265a143ef8244135b80c68c0c', 'request_id': 'req-810443d6-1e94-483b-99b6-2ee77282ad55'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:57.913375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c7c1e265a143ef8244135b80c68c0c', 'request_id': 'req-810443d6-1e94-483b-99b6-2ee77282ad55'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:57.953389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] The token cache is being invalidate because project f2c7c1e265a143ef8244135b80c68c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:57.954045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:57.954973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:57.962824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 966/1934] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:57 2022] DELETE /identity/v3/projects/f2c7c1e265a143ef8244135b80c68c0c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:57.965047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.969863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.000423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f725-a463-490f-a324-abfb2b1eb437 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.096316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810443d6-1e94-483b-99b6-2ee77282ad55 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.104250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.105415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.105708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.105986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] PATH_INFO: `/v3/users/19b9ddc7f27c4938b7d5e1251da2e73d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.107790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b9ddc7f27c4938b7d5e1251da2e73d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.112071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.145609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:58.149885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.150669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.151465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.151862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.152743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.154549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.174253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 969/1935] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:58.177639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:58.180297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.204800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b9ddc7f27c4938b7d5e1251da2e73d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.209166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3730ef-e842-43be-b176-85c3aef9b49e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.216572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.217481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.217734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.218031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] PATH_INFO: `/v3/users/ed5a6f71105d4201afd740ae6f428cba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.219281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5a6f71105d4201afd740ae6f428cba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.221333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.227461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b9ddc7f27c4938b7d5e1251da2e73d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.251782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b9ddc7f27c4938b7d5e1251da2e73d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.253377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b9ddc7f27c4938b7d5e1251da2e73d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.276044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b9ddc7f27c4938b7d5e1251da2e73d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.285535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 967/1936] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:57 2022] DELETE /identity/v3/users/19b9ddc7f27c4938b7d5e1251da2e73d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:58.286749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:58.288802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.327768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5a6f71105d4201afd740ae6f428cba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.329810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:58.355069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5a6f71105d4201afd740ae6f428cba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.370603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5a6f71105d4201afd740ae6f428cba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.371846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5a6f71105d4201afd740ae6f428cba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.373093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7dfdef53-35b5-46ae-b83b-1cf01a8504c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5a6f71105d4201afd740ae6f428cba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.390846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156c73-5646-4599-8a1b-66cc8343e534 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.399808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 970/1937] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:58 2022] DELETE /identity/v3/users/ed5a6f71105d4201afd740ae6f428cba => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:58.401071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.402104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.402463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.403121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] PATH_INFO: `/v3/projects/1dc2295f232841798892910f44d3281c/users/83b277de18514ab7ab0dee616ac1df5b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.405187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc2295f232841798892910f44d3281c, user_id=83b277de18514ab7ab0dee616ac1df5b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.405858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.406363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.407239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.408611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6c60e-f820-4105-9360-3dd4b2413edd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.421610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 968/1938] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:58 2022] PUT /identity/v3/projects/1dc2295f232841798892910f44d3281c/users/83b277de18514ab7ab0dee616ac1df5b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:58.435833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.436860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.437030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:58.437191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.438063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.484712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] MFA Rules not processed for user `83b277de18514ab7ab0dee616ac1df5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:58.485957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.550755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 971/1939] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:58.565858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:58.567406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.620839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 969/1940] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:58.634120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:58.637130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.656850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757c76a-81ec-40c3-8d68-c62510b5e8f7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.668445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.669318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.669698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.672713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.673088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:58.699404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.702106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.716823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 972/1941] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:58.717768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:58.719427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:58.742592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039cba-de51-495a-8736-927bfc5371bd service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.750704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029b1a2-5724-42e9-95ec-bd290d0c9c36 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:58.756533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.758736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.759232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.759510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.761299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:58.762126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:58.762551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:58.762981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] PATH_INFO: `/v3/projects/a7c2f09e8d544951bce8f9747db12c47` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:58.764439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c2f09e8d544951bce8f9747db12c47)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.766662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.774078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:58.776206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35d6d0-fdc2-41ce-9eca-262a8cce1945 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:58.791643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 970/1942] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:58.918238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c2f09e8d544951bce8f9747db12c47', 'request_id': 'req-9b131583-5c76-4729-81d9-f3df1939c222'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.930444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c2f09e8d544951bce8f9747db12c47', 'request_id': 'req-9b131583-5c76-4729-81d9-f3df1939c222'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.938457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c2f09e8d544951bce8f9747db12c47', 'request_id': 'req-9b131583-5c76-4729-81d9-f3df1939c222'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.950258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:58.990577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] The token cache is being invalidate because project a7c2f09e8d544951bce8f9747db12c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:58.990814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.991599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9b131583-5c76-4729-81d9-f3df1939c222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:58.994738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 973/1943] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:58 2022] DELETE /identity/v3/projects/a7c2f09e8d544951bce8f9747db12c47 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:58.999886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.000174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.000454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.044309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:59.046989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:59.201001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 971/1944] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:59.208072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.208353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.208657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.259573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:59.261687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:59.276640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 974/1945] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:59.278510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.297034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdb7ef-da92-434c-a024-faf0d5c598f8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.313684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.314585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.315252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.317115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.317827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:59.318140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.319775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.344810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 972/1946] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:59.347016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.366956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf588a9-8fac-4b5c-8eb5-6b4882625e1f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.379716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.380363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.380670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.380947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.381963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.384779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.438899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 975/1947] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:59 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:59.444104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.462172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d9c2b-933b-49ff-bbd9-aa5f407b49f6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.485226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.488391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:59.627335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.631542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.631858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.632167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.634452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.637624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.651404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc23bba-3746-4677-bd53-04754a1868cc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.656984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.657662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.658010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.658292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] PATH_INFO: `/v3/users/379dcb04a9e046259508a83ef1424274` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.659761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=379dcb04a9e046259508a83ef1424274)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.662133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.669994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 973/1948] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:59 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:59.673899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.684185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bbcd8-3152-4181-8645-7baaf74c7f19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.691742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.691742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.691742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.691742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.692128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.693088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.712338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 974/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:59.712338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.723054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379dcb04a9e046259508a83ef1424274'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:59.731807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e825b81-40b0-449e-888a-eb99b49892c4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.738771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.739393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.739685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.739991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.740673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379dcb04a9e046259508a83ef1424274'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:59.740803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.742460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.755108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379dcb04a9e046259508a83ef1424274'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:59.756516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379dcb04a9e046259508a83ef1424274'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:59.757349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379dcb04a9e046259508a83ef1424274'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:59.762181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 975/1950] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:59.763704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.772141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:59.790972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 976/1951] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:59 2022] DELETE /identity/v3/users/379dcb04a9e046259508a83ef1424274 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:59.791899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:59.793480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:59.808959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824747-1520-4136-ba93-ca8a1db2bbc0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.834597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:59.892468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:59.932193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66808a-cc58-43f2-9453-6dc4c32f4bfc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:59.939207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.940925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.941509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.941825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] PATH_INFO: `/v3/projects/ca49c2e79b39471e9927123e20cf6658/users/6ccf83d4cee740d4b1dee835cb4741ca/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.966089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:59.967320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.967433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca49c2e79b39471e9927123e20cf6658, user_id=6ccf83d4cee740d4b1dee835cb4741ca, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.967931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.968298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] PATH_INFO: `/v3/projects/2d4a4d67c85740d09e766ad2fc14952a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:59.970234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4a4d67c85740d09e766ad2fc14952a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:59.970581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621bc73-f999-472f-acaf-045dc6c393ae admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.977134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:59.988945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 977/1952] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:59 2022] PUT /identity/v3/projects/ca49c2e79b39471e9927123e20cf6658/users/6ccf83d4cee740d4b1dee835cb4741ca/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:59.990856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35577ca-97c2-4b59-8a23-720cebd5e461 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:59.991226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35577ca-97c2-4b59-8a23-720cebd5e461 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:59.991723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35577ca-97c2-4b59-8a23-720cebd5e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.015535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c35577ca-97c2-4b59-8a23-720cebd5e461 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:00.017051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c35577ca-97c2-4b59-8a23-720cebd5e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.093209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4a4d67c85740d09e766ad2fc14952a', 'request_id': 'req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.117076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4a4d67c85740d09e766ad2fc14952a', 'request_id': 'req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.130992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4a4d67c85740d09e766ad2fc14952a', 'request_id': 'req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.148127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 978/1953] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:00.160397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.160575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.160960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.161057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] The token cache is being invalidate because project 2d4a4d67c85740d09e766ad2fc14952a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:00.161305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.161987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.165346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 976/1954] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:59 2022] DELETE /identity/v3/projects/2d4a4d67c85740d09e766ad2fc14952a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:00.169583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:00.177700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.213536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] MFA Rules not processed for user `6ccf83d4cee740d4b1dee835cb4741ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:00.214928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.319433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60a4f6-5c5d-47d5-9ad7-1d1a9075c9a4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:00.325489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:00.326681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.327298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.327873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.331553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.375859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 979/1955] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:00.375859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:00.388119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.487223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:00.497746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:00.499214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:00.512655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:00.522512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebfd29-5c51-4686-860e-bcd429d0e3c9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:00.528989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 977/1956] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:00.529832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:00.533168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:00.533951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.535068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.536301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.536528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] PATH_INFO: `/v3/users/a0887eb44270424bb6ed65078b454006` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.539106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0887eb44270424bb6ed65078b454006)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:00.541415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:00.571350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f792-d137-4737-b4ef-428d3125daf1 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:00.578258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:00.579447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.580019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.580344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.581113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:00.583394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:00.641068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 978/1957] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:00 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:00.647345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:00.649570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0887eb44270424bb6ed65078b454006'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.680935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0887eb44270424bb6ed65078b454006'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.706656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0887eb44270424bb6ed65078b454006'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.708017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0887eb44270424bb6ed65078b454006'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.711744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0887eb44270424bb6ed65078b454006'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:00.740024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 980/1958] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:00 2022] DELETE /identity/v3/users/a0887eb44270424bb6ed65078b454006 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:00.787421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7ef43-9856-43b6-b3eb-c4cc942bff72 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:00.905677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:00.906523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.906954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.907234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.907883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:00.910502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:00.940634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 979/1959] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:00 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:00.945244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:00.947000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:00.961380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11252d81-4264-4271-9200-55c1b0f5b546 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:00.967713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:00.968534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:00.968707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:00.969077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:00.969829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:00.972149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:00.984575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 981/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:00.988808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.001593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b69eb0-13ef-4418-a67e-545d973ed04b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.008284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.009300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.009486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.009747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] PATH_INFO: `/v3/projects/06648917f7d546a8ab0cda25c7c21dcd/users/ccc93a41b4f04704a7de815ea3f6cbad/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.034706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06648917f7d546a8ab0cda25c7c21dcd, user_id=ccc93a41b4f04704a7de815ea3f6cbad, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.037559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.053391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 980/1961] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:00 2022] PUT /identity/v3/projects/06648917f7d546a8ab0cda25c7c21dcd/users/ccc93a41b4f04704a7de815ea3f6cbad/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:01.057539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.070944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23719613-75fb-4bf6-a6e8-b6d54e2d15cf admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.165204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.166247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.166730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.167201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.168160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.170245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab1e01-36d5-471d-8940-298b5280e2c1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.189163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 982/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:01.189879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.201238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3db38-385d-44e0-bbc7-88db58bb0e57 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.207304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.208017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.208300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.208619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] PATH_INFO: `/v3/projects/06648917f7d546a8ab0cda25c7c21dcd/users/ccc93a41b4f04704a7de815ea3f6cbad/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.214014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06648917f7d546a8ab0cda25c7c21dcd, user_id=ccc93a41b4f04704a7de815ea3f6cbad, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.216710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.259639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 981/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:01 2022] PUT /identity/v3/projects/06648917f7d546a8ab0cda25c7c21dcd/users/ccc93a41b4f04704a7de815ea3f6cbad/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:01.270581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.270581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.270861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.308945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] MFA Rules not processed for user `ccc93a41b4f04704a7de815ea3f6cbad`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:01.310875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.381514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:01.441978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 983/1964] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:01.458891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.460635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.541807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:01.576430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b548a-78e3-4201-9194-4dfe417a6686 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.591251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.592818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.598015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.598875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.598970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.599304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.604846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.707232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df178dbc-0281-482f-a3da-51aff76ecf07 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.712549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.713378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.713782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.714287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] PATH_INFO: `/v3/projects/0448a001b4714952b7d9767ec2d2cacc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.717357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0448a001b4714952b7d9767ec2d2cacc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.720032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.730086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.738675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.762174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 982/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:01.763920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.766052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.790367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c0042-c121-4616-bbeb-d5cf77c07225 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.820500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0448a001b4714952b7d9767ec2d2cacc', 'request_id': 'req-55e3fb11-bced-40ac-b687-c76aba376c78'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.842445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0448a001b4714952b7d9767ec2d2cacc', 'request_id': 'req-55e3fb11-bced-40ac-b687-c76aba376c78'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.860675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0448a001b4714952b7d9767ec2d2cacc', 'request_id': 'req-55e3fb11-bced-40ac-b687-c76aba376c78'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.871658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:01.872483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:01.872802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:01.873085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] PATH_INFO: `/v3/projects/867d22d866ef4f38a5101b4e1f7b3010` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:01.874527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867d22d866ef4f38a5101b4e1f7b3010)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:01.877531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:01.884437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] The token cache is being invalidate because project 0448a001b4714952b7d9767ec2d2cacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:01.884790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.886443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.895981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 984/1966] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:01 2022] DELETE /identity/v3/projects/0448a001b4714952b7d9767ec2d2cacc => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:01.906079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:01.911847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:01.961782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fb11-bced-40ac-b687-c76aba376c78 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:01.963877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867d22d866ef4f38a5101b4e1f7b3010', 'request_id': 'req-5a5ba058-c233-4c4f-83dd-ae922794ef12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.974018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867d22d866ef4f38a5101b4e1f7b3010', 'request_id': 'req-5a5ba058-c233-4c4f-83dd-ae922794ef12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:01.993837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867d22d866ef4f38a5101b4e1f7b3010', 'request_id': 'req-5a5ba058-c233-4c4f-83dd-ae922794ef12'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.022144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] The token cache is being invalidate because project 867d22d866ef4f38a5101b4e1f7b3010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:02.022144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.022636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5a5ba058-c233-4c4f-83dd-ae922794ef12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.024470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 983/1967] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:01 2022] DELETE /identity/v3/projects/867d22d866ef4f38a5101b4e1f7b3010 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:02.029358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f27bec-e2a8-40ce-a8cf-8c45af8c10c9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.030330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f27bec-e2a8-40ce-a8cf-8c45af8c10c9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.030623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f27bec-e2a8-40ce-a8cf-8c45af8c10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.065710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c1f27bec-e2a8-40ce-a8cf-8c45af8c10c9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:02.074787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c1f27bec-e2a8-40ce-a8cf-8c45af8c10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:02.099343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:02.117099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.118627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.119004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.119313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] PATH_INFO: `/v3/users/83b277de18514ab7ab0dee616ac1df5b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.120825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b277de18514ab7ab0dee616ac1df5b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.122841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:02.181558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:02.232097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 984/1968] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:02.241222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.241222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.241581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.259853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b277de18514ab7ab0dee616ac1df5b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.286640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:02.289326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:02.289458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b277de18514ab7ab0dee616ac1df5b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.310478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b277de18514ab7ab0dee616ac1df5b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.312985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 985/1969] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:02.317155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:02.317465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b277de18514ab7ab0dee616ac1df5b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.318291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b277de18514ab7ab0dee616ac1df5b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:02.335416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31ff2-e3f9-41f3-9fd4-cdf2d906779c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:02.346946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.347788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.348065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.348343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.349030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:02.349334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.350957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:02.354968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 985/1970] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:01 2022] DELETE /identity/v3/users/83b277de18514ab7ab0dee616ac1df5b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:02.379539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 986/1971] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:35:02.386999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:02.389256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:02.572959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6b7d-2f53-4378-aa9d-ebc86ed3df9b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:02.584825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.586377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.587764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.588871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.589755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.607660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:02.688630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 986/1972] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:02.692818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:02.708371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a4ba-fc04-4971-a150-8892096e299d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:02.863820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.866608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.866608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.867450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.867758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.869848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:02.907912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 987/1973] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:02 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:02.909511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:02.920914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fe99d-e561-4387-b08d-e21929c1d98b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:02.928071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.928902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.929127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.930001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.930240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.931901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:02.947668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 987/1974] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:02.953005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:02.969429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2069-82e1-4944-9c58-12ad1a5098e2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:02.977114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:02.977754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:02.978035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:02.978369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:02.978964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:02.985855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf94a1f-fb44-4155-813d-f382c33894f5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.018845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 988/1975] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:03.023617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.041743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1bbb2-f3fe-4b61-a4ce-7924cdff5a42 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.061494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.062722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.063189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.064983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] PATH_INFO: `/v3/projects/7faae8d00a06427983b2944792166135/users/66df13267c5f4b14abb31f1bd88241d9/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.109467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7faae8d00a06427983b2944792166135, user_id=66df13267c5f4b14abb31f1bd88241d9, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.112774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.147648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 988/1976] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:03 2022] PUT /identity/v3/projects/7faae8d00a06427983b2944792166135/users/66df13267c5f4b14abb31f1bd88241d9/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:03.152488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.152949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.153337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.183381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:03.184429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.256616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.262945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.320231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 989/1977] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:03.325691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.339130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.408389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:03.432041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:03.477417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8169e-c03c-42d5-92d2-34e54d29d7a5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.490760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.492214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.492214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.492519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] PATH_INFO: `/v3/projects/1dc2295f232841798892910f44d3281c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.495468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc2295f232841798892910f44d3281c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.511073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103601-510a-4031-be4d-9caef03ab3cd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.513244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.515943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.516682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.517020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.517297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.531319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.537048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd4720-6782-4ff5-a48a-eed5e62f8c44 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.550106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 990/1978] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:03.588731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc2295f232841798892910f44d3281c', 'request_id': 'req-ca57821e-d70b-4adb-8c14-f92427684d1a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.598400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc2295f232841798892910f44d3281c', 'request_id': 'req-ca57821e-d70b-4adb-8c14-f92427684d1a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.604320 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc2295f232841798892910f44d3281c', 'request_id': 'req-ca57821e-d70b-4adb-8c14-f92427684d1a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.620298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] The token cache is being invalidate because project 1dc2295f232841798892910f44d3281c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:03.620735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.621672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ca57821e-d70b-4adb-8c14-f92427684d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.624512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 989/1979] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:03 2022] DELETE /identity/v3/projects/1dc2295f232841798892910f44d3281c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:03.629756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.630046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.630874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.675027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:03.676321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.779782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 991/1980] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:03.787529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.787839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.788097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.812671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:03.814867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.833165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 990/1981] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:03.834932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.850490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b74874-9536-4ff8-89aa-aa974c1b7669 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.859898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.860983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.861736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.862351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.869069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:03.870220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.873261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.895122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 992/1982] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:35:03.900878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.917475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85824e02-dfed-49bf-ad44-f6f9f8b7344f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.936705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.937606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.937972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.938477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.939324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.942368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.004340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.005227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 991/1983] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:03 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:04.017123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01fe4c-3786-49bc-9e9d-d286a9d4bcdb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.155795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.156763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.157188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.157603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.158387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.161056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.200344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 993/1984] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:04 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:04.201634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.214190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037cf2-d82d-466e-afc6-472c8cd11c3f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.220493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.222053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.222551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.222966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.223758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.225895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.245994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 992/1985] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:04.247596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.261359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfc69c-e9e9-42c7-a267-9a0be26c29a6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.266986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.267587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.267867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.268262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.269991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.271395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fa71e-dd51-460b-82dd-d300639a47b1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.284177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 994/1986] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:04.289318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.301563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b09106-08f7-4224-8c88-3c0fe0578bab admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.307250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.307446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.307570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.307893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] PATH_INFO: `/v3/projects/e6b5d2fe226f4e7d843bdbb75c798145/users/e255245a67a343b0927a316c9575682a/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.324697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b5d2fe226f4e7d843bdbb75c798145, user_id=e255245a67a343b0927a316c9575682a, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.327718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.344149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 993/1987] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:04 2022] PUT /identity/v3/projects/e6b5d2fe226f4e7d843bdbb75c798145/users/e255245a67a343b0927a316c9575682a/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:04.356507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.356836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.357107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.388224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] MFA Rules not processed for user `e255245a67a343b0927a316c9575682a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:04.389491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:04.403172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:04.464736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 995/1988] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:04.469161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.547671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d9ed2-0f03-4c43-ad0f-e584992bebe4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.555648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.556533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.557923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.558233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.558913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.560955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.576835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:04.591175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 994/1989] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:04 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:04.596301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.607568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886d45c-4318-4677-bbdb-eb0a56593052 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.667139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.667755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.668037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.668325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.668991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.670894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.693548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 996/1990] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:04 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:04.697577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.708746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2dc88-fdce-4a34-b96c-8cd5cb91d69b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.715753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.717586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.718042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.718483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.719622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.722379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.739589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 995/1991] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:04.743682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.757682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66fe6a-5d75-4f26-b419-4c2ae088a769 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.764402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.765032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.765354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.765632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] PATH_INFO: `/v3/projects/dd566edb2d574864853f3ccc021296b2/users/f7a9f83ae0ba44bbb9655439c6aaa3e3/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.792879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd566edb2d574864853f3ccc021296b2, user_id=f7a9f83ae0ba44bbb9655439c6aaa3e3, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.796440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c76506-4032-4968-bb57-b76c5fd3027f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.815807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 997/1992] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:04 2022] PUT /identity/v3/projects/dd566edb2d574864853f3ccc021296b2/users/f7a9f83ae0ba44bbb9655439c6aaa3e3/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:04.820205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.830945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8134444-cce3-4f90-820f-c812818e22ed admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.922105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.923448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.923448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.923676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.924205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.926163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.937034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2462-bc56-499b-b832-fa65e5261257 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.937585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2462-bc56-499b-b832-fa65e5261257 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.937817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2462-bc56-499b-b832-fa65e5261257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.950319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 996/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:04.958066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.974799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b0d1e-660b-4a0a-81d9-aa4e88598f01 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.980902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.981853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.982144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.982419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] PATH_INFO: `/v3/projects/dd566edb2d574864853f3ccc021296b2/users/f7a9f83ae0ba44bbb9655439c6aaa3e3/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.984656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd566edb2d574864853f3ccc021296b2, user_id=f7a9f83ae0ba44bbb9655439c6aaa3e3, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.987577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.002931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 997/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:04 2022] PUT /identity/v3/projects/dd566edb2d574864853f3ccc021296b2/users/f7a9f83ae0ba44bbb9655439c6aaa3e3/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:05.004065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.007916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.010096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-424c2462-bc56-499b-b832-fa65e5261257 None None] MFA Rules not processed for user `66df13267c5f4b14abb31f1bd88241d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:05.011535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-424c2462-bc56-499b-b832-fa65e5261257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.101751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c32a2-5073-44ff-a8e8-023b1cf85c64 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.121524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.122438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.123046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.123418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] PATH_INFO: `/v3/users/6ccf83d4cee740d4b1dee835cb4741ca` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.124884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccf83d4cee740d4b1dee835cb4741ca)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.127061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.145385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 998/1995] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:05.147455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914059-ca8e-4f56-aeb3-c7e0c1f16887 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.147862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914059-ca8e-4f56-aeb3-c7e0c1f16887 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.148264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7914059-ca8e-4f56-aeb3-c7e0c1f16887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.174130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:05.185235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f7914059-ca8e-4f56-aeb3-c7e0c1f16887 None None] MFA Rules not processed for user `f7a9f83ae0ba44bbb9655439c6aaa3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:05.186678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f7914059-ca8e-4f56-aeb3-c7e0c1f16887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.210222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccf83d4cee740d4b1dee835cb4741ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.240514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccf83d4cee740d4b1dee835cb4741ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.249273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:05.251640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccf83d4cee740d4b1dee835cb4741ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.253103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccf83d4cee740d4b1dee835cb4741ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.254978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccf83d4cee740d4b1dee835cb4741ca'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.280211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 998/1996] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:05 2022] DELETE /identity/v3/users/6ccf83d4cee740d4b1dee835cb4741ca => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:05.280211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.280748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.294390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 999/1997] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:05.296076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.296076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.296323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.322732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:05.324072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.365938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e90d-cca3-4a07-84c8-bd5f1f53d3dc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.371021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.371760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.372135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.372502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.373258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.375240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.396872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1000/1998] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:05.397927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.399431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.411807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 999/1999] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:05 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:05.413353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.416645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.462795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e21b14-5818-4f0f-8ab6-a7dd7ae07ced None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.485504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74811c99-1289-4c62-a34a-7ec1268a96a5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.490032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.490573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.491343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.492207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.520000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.520689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.520975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.521251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.523173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.530528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:05.578433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.579910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.608012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1000/2000] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:05.610002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.622234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.625083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.637734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4707319-14e9-40fd-8b6c-b1eb38d1f540 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.644573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.645560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.645849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.646185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.646848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.649891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.653088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1001/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:05.682611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1001/2002] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:05 2022] POST /identity/v3/users => generated 292 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:05.684077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.696806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da15-3b5f-49a7-9642-3b3a35195748 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.704852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.705881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.706405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.706789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.707595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.709459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.731535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1002/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:05.735151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.744086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:05.749566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2431d-9fef-4282-857c-bef6463e2943 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.758239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.759053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.759553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.759951 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] PATH_INFO: `/v3/projects/bf2f032d6edd45349370a7d99dc4ee78/users/6a0c5afd54634878a168954cb2d5bcaf/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.778816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2f032d6edd45349370a7d99dc4ee78, user_id=6a0c5afd54634878a168954cb2d5bcaf, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.784847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.810616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1002/2004] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:05 2022] PUT /identity/v3/projects/bf2f032d6edd45349370a7d99dc4ee78/users/6a0c5afd54634878a168954cb2d5bcaf/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:05.816157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.830672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d28a1-0370-4f64-992e-045a9fbd3a36 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.907211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.908037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.908437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.908930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.911301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.914274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.929277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1003/2005] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:05.933984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.959361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0e8b7-85f6-4c5b-82dc-021a6e0421f6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.978232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.981532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.982357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.982656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] PATH_INFO: `/v3/projects/bf2f032d6edd45349370a7d99dc4ee78/users/6a0c5afd54634878a168954cb2d5bcaf/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.989322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2f032d6edd45349370a7d99dc4ee78, user_id=6a0c5afd54634878a168954cb2d5bcaf, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.992083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.010343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1003/2006] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:05 2022] PUT /identity/v3/projects/bf2f032d6edd45349370a7d99dc4ee78/users/6a0c5afd54634878a168954cb2d5bcaf/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:06.403060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.405194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:06.424965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d36eb3-ca77-4ad6-a2e5-8f3a27360a86 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.530309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.531452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.531858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.532286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] PATH_INFO: `/v3/projects/ca49c2e79b39471e9927123e20cf6658` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.534042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca49c2e79b39471e9927123e20cf6658)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.536914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.590602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:06.608098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca49c2e79b39471e9927123e20cf6658', 'request_id': 'req-796de1df-1485-4314-9064-00e00b737d31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.625440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca49c2e79b39471e9927123e20cf6658', 'request_id': 'req-796de1df-1485-4314-9064-00e00b737d31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.642017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca49c2e79b39471e9927123e20cf6658', 'request_id': 'req-796de1df-1485-4314-9064-00e00b737d31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.661559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] The token cache is being invalidate because project ca49c2e79b39471e9927123e20cf6658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:06.662107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.663312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-796de1df-1485-4314-9064-00e00b737d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.666019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1004/2007] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:06 2022] DELETE /identity/v3/projects/ca49c2e79b39471e9927123e20cf6658 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:06.669847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.671870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:06.687530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923bd81-4941-4ef6-ae08-e27bb0c092a7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.787796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.788904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.790094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.790640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] PATH_INFO: `/v3/users/ccc93a41b4f04704a7de815ea3f6cbad` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.794039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc93a41b4f04704a7de815ea3f6cbad)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.797742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.906730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc93a41b4f04704a7de815ea3f6cbad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.922400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc93a41b4f04704a7de815ea3f6cbad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.933351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc93a41b4f04704a7de815ea3f6cbad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.935326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc93a41b4f04704a7de815ea3f6cbad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.935326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc93a41b4f04704a7de815ea3f6cbad'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:06.954676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1004/2008] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:06 2022] DELETE /identity/v3/users/ccc93a41b4f04704a7de815ea3f6cbad => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:07.703069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.703647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.704611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.765363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] MFA Rules not processed for user `6a0c5afd54634878a168954cb2d5bcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:07.766627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:07.925018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.926457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:07.940553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1005/2009] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:07.958688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.960154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:07.999653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:08.024317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b31904-b8c3-4b1f-868c-2f97c72c3aad admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.030877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.031661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.032416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.032416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] PATH_INFO: `/v3/projects/06648917f7d546a8ab0cda25c7c21dcd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.034262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06648917f7d546a8ab0cda25c7c21dcd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.036465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.074255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c7a14-b9ff-419d-90f4-59885ea50dd8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.088638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.089664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.090133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.091754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.094670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:08.108468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06648917f7d546a8ab0cda25c7c21dcd', 'request_id': 'req-f5b32b6e-a185-4a75-ada4-76d589902878'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.120079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06648917f7d546a8ab0cda25c7c21dcd', 'request_id': 'req-f5b32b6e-a185-4a75-ada4-76d589902878'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.127640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06648917f7d546a8ab0cda25c7c21dcd', 'request_id': 'req-f5b32b6e-a185-4a75-ada4-76d589902878'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.141794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] The token cache is being invalidate because project 06648917f7d546a8ab0cda25c7c21dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:08.142110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.143608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f5b32b6e-a185-4a75-ada4-76d589902878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.145431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1005/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:07 2022] DELETE /identity/v3/projects/06648917f7d546a8ab0cda25c7c21dcd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:08.150639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.151028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.151237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.172941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:08.174815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:08.185179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.186943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a7417-1845-4455-8194-8e01e075c232 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.199093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1006/2011] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:08.253201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1006/2012] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:08.258778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.259120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.259457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.269255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:08.294221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:08.295674 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:08.306503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1007/2013] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:08.310787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.325214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcabd6-3dcd-4160-83f7-be8805361979 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.341098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.341728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.342006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.342284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.342944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:08.343251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.344982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.363083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1007/2014] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:08.370738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.383435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ae54-4941-4020-b83a-d7953569d7b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.389377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.390006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.390283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.390558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.391209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.405660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.470205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1008/2015] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:08 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:08.475538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.494035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e0b19-4763-4f57-901d-8cba88318506 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.612815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.613451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.613728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.614003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.614653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.618324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.630331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:08.652017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1008/2016] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:08 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:08.655979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.676000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9c26-41dc-4daa-94cf-19dda115a82e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.681988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.682805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.683098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.683379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.684055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.685855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.699489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1009/2017] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:08.703293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.717463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfe920-8c6d-460f-b579-b456d91d17ae admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.723527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.724460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.724864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.725271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.726150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.728356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7052-9aa9-494f-b272-387b11ad8083 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.746516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1009/2018] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:08.751426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.769456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb90f70-17d9-4a83-baf3-1597cc949355 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.781469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.784201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.785164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.785743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] PATH_INFO: `/v3/projects/e09c39e301ad4cb2b7dd354345ab8d5a/users/75a1028196964360b4fa3b57950c360f/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.807887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09c39e301ad4cb2b7dd354345ab8d5a, user_id=75a1028196964360b4fa3b57950c360f, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.810898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.838904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1010/2019] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:08 2022] PUT /identity/v3/projects/e09c39e301ad4cb2b7dd354345ab8d5a/users/75a1028196964360b4fa3b57950c360f/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:08.850869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.851160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.851439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.904622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] MFA Rules not processed for user `75a1028196964360b4fa3b57950c360f`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:08.904622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.012999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1010/2020] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:09.014688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.124215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbb6d5-3614-4d66-955a-ed35cfc28126 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.131345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.132358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.132905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.146014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.146014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.147486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.213859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1011/2021] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:09.219456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.241804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74fd53-048b-4de1-815c-651db3d981a9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.350753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.352871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.353785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.354258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.355144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.358561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.397243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1011/2022] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:09.398945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.411661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0498059-4c21-4334-87bb-a73473212de4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.417370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.418143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.418436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.418710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.419373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.421036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.436380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1012/2023] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:09.437697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.454448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3950b-bab9-4ecf-a588-5967e3b89182 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.461690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.463136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.463766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.464327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.465386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.467501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d978e-ba25-4335-8b38-70ac382baad3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.488411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1012/2024] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:09.491570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.510163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1dad6e-9065-462a-946b-c870eb79cf31 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.518569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.520165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.520706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.521214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] PATH_INFO: `/v3/projects/d973e12634da4b388cea66eaf24f9878/users/a9e80d2b4e9e43b7b1e98c3b7af82088/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.551290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d973e12634da4b388cea66eaf24f9878, user_id=a9e80d2b4e9e43b7b1e98c3b7af82088, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.556011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.575210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1013/2025] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:09 2022] PUT /identity/v3/projects/d973e12634da4b388cea66eaf24f9878/users/a9e80d2b4e9e43b7b1e98c3b7af82088/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:09.586400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.586696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.586976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.620656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] MFA Rules not processed for user `a9e80d2b4e9e43b7b1e98c3b7af82088`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:09.621898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.693763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:09.732746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1013/2026] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:09.733375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.753716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.755229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.834304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:09.860310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe46e5-2e4f-4095-ba0a-d7b34a201b6a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.866373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.868540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.868873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.869764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.870433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.871326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad63f0-3645-4b78-bc53-b906bff044d1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.872691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.879721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.881161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.881890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.882649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.887718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.928367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1014/2027] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:09.935145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.018929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158ea47-a6f6-41c9-83e6-528a74f2ef2f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.029251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.029886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.030738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.030738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.031051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.035020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.055687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.058614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.062575 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1015/2028] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:10.073871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.075620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1014/2029] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:10.093984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761a91b-44ff-426f-a4d3-eac870800592 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.100428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.100966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.101341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.101624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.102299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.103995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.118055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1016/2030] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:10.122893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.143318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960594c-fc63-4929-90cb-2d72511f2995 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.151627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.153242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.154130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.154594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] PATH_INFO: `/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9/users/a88c7eafd39945a9bd03cebab3fcf352/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.182542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ebdc2810e641f4a8b6229ff259a6a9, user_id=a88c7eafd39945a9bd03cebab3fcf352, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.190429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.207476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1015/2031] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:10 2022] PUT /identity/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9/users/a88c7eafd39945a9bd03cebab3fcf352/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:10.211685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.225276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b452ae0-c04e-4207-99d0-de728af62d0c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.304048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.305206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.305206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.305576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.306103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.307719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd38a39-ba17-47ff-b2f1-a7d99e9ce850 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.318713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1017/2032] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:10.323248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.336343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338dcdc-f9ec-4d70-8315-48de6f070bfd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.341480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.342151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.342430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.342857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] PATH_INFO: `/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9/users/a88c7eafd39945a9bd03cebab3fcf352/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.345480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ebdc2810e641f4a8b6229ff259a6a9, user_id=a88c7eafd39945a9bd03cebab3fcf352, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.348944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.367860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1016/2033] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:10 2022] PUT /identity/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9/users/a88c7eafd39945a9bd03cebab3fcf352/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:10.378880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.379476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.379937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.447637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] MFA Rules not processed for user `a88c7eafd39945a9bd03cebab3fcf352`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:10.449185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:10.507340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:10.547313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.549701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1018/2034] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:10.562408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.563838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:10.635646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b33210-629f-4221-97c5-6cef4dbd2dfe admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.636076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cc93d-9154-4a05-b85e-50a46eaab3b7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.640048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.640688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.640974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.641240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.641531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.642323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.642966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.643310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:10.643462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.653423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:10.722902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.726861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.730941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.732913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.738692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1019/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:10.746676 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1017/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:12.446168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.460066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516105a1-553a-4ea7-9e29-a6bd228e592c service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.464506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.465222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.465857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.466200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.533042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.535036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.544137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1020/2037] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:13.224199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:13.233889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0780350-8746-4da3-9392-ca18150d6e74 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:13.237642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:13.238255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:13.238536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:13.238808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:13.248316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:13.250050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536b8b5-da1e-4f7d-a3ce-f77dae4bb7d7 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:13.263126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1018/2038] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:15.905851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:15.907804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:15.943501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:16.004138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d977bca-b503-468a-b6f8-a18193e42d5e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:16.010935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:16.011723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.012091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.012484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] PATH_INFO: `/v3/users/e255245a67a343b0927a316c9575682a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.013984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e255245a67a343b0927a316c9575682a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:16.023405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:16.094831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e255245a67a343b0927a316c9575682a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.107117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e255245a67a343b0927a316c9575682a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.125906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e255245a67a343b0927a316c9575682a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.148697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e255245a67a343b0927a316c9575682a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.186254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e255245a67a343b0927a316c9575682a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.194440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1021/2039] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:15 2022] DELETE /identity/v3/users/e255245a67a343b0927a316c9575682a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:16.199068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.200054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.200355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.222453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:16.223871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:16.304185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1019/2040] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:16.315500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:16.317451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:16.399269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066fb2f-5ed4-4854-806f-664a7f786c94 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:16.405598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:16.406407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.406689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.406967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.416825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:16.418276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:16.433328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1022/2041] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:16.750765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:16.753038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:16.779112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d29cd9-2873-4b8d-ad9a-0e42957351b8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:16.804349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:16.805053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.805345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.805692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] PATH_INFO: `/v3/projects/e6b5d2fe226f4e7d843bdbb75c798145` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.807076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b5d2fe226f4e7d843bdbb75c798145)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:16.809137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:16.906559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b5d2fe226f4e7d843bdbb75c798145', 'request_id': 'req-fceda910-d9c1-4be3-ba24-2da1ca75cc24'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.920044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b5d2fe226f4e7d843bdbb75c798145', 'request_id': 'req-fceda910-d9c1-4be3-ba24-2da1ca75cc24'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.929099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b5d2fe226f4e7d843bdbb75c798145', 'request_id': 'req-fceda910-d9c1-4be3-ba24-2da1ca75cc24'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.942252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] The token cache is being invalidate because project e6b5d2fe226f4e7d843bdbb75c798145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:16.942504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.943433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.945104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1020/2042] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:16 2022] DELETE /identity/v3/projects/e6b5d2fe226f4e7d843bdbb75c798145 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:16.949316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:16.961245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754ef09-29d9-4154-9355-2fe5715cb532 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.016517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:17.034091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.034625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.034910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.035212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] PATH_INFO: `/v3/users/f7a9f83ae0ba44bbb9655439c6aaa3e3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.036337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a9f83ae0ba44bbb9655439c6aaa3e3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.038332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.125371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a9f83ae0ba44bbb9655439c6aaa3e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.136181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a9f83ae0ba44bbb9655439c6aaa3e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.145804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a9f83ae0ba44bbb9655439c6aaa3e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.146664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a9f83ae0ba44bbb9655439c6aaa3e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.164512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d27f06c5-bcfc-4172-a52b-04f5a660c13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a9f83ae0ba44bbb9655439c6aaa3e3'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.184898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1023/2043] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:16 2022] DELETE /identity/v3/users/f7a9f83ae0ba44bbb9655439c6aaa3e3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:17.285399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.287552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.311262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceda910-d9c1-4be3-ba24-2da1ca75cc24 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.324762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:17.397461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.397685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.397954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.398249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] PATH_INFO: `/v3/users/75a1028196964360b4fa3b57950c360f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.399448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a1028196964360b4fa3b57950c360f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.401430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.454343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a1028196964360b4fa3b57950c360f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.468888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a1028196964360b4fa3b57950c360f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.477749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a1028196964360b4fa3b57950c360f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.478545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a1028196964360b4fa3b57950c360f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.479295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a1028196964360b4fa3b57950c360f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.495341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1021/2044] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:17 2022] DELETE /identity/v3/users/75a1028196964360b4fa3b57950c360f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:17.500443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2793-8f0f-4953-a111-5919c537420d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.500744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2793-8f0f-4953-a111-5919c537420d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.501021 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2793-8f0f-4953-a111-5919c537420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.510783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.512909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.523886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-91db2793-8f0f-4953-a111-5919c537420d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:17.525426 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-91db2793-8f0f-4953-a111-5919c537420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.595144 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1024/2045] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:17.598728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58502-3e4b-4172-a27a-4519feb9c4b5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.603789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.604423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.604681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.605089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] PATH_INFO: `/v3/projects/dd566edb2d574864853f3ccc021296b2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.606306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd566edb2d574864853f3ccc021296b2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.607486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2793-8f0f-4953-a111-5919c537420d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.608654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.608944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-91db2793-8f0f-4953-a111-5919c537420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.649170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:17.661350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd566edb2d574864853f3ccc021296b2', 'request_id': 'req-d5da286b-dd90-498a-b40c-bb0bde87467e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.668909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd566edb2d574864853f3ccc021296b2', 'request_id': 'req-d5da286b-dd90-498a-b40c-bb0bde87467e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.672425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2793-8f0f-4953-a111-5919c537420d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.676324 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd566edb2d574864853f3ccc021296b2', 'request_id': 'req-d5da286b-dd90-498a-b40c-bb0bde87467e'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.688789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] The token cache is being invalidate because project dd566edb2d574864853f3ccc021296b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:17.689062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.690250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.695849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:17.701894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1022/2046] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:17 2022] DELETE /identity/v3/projects/dd566edb2d574864853f3ccc021296b2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:17.734036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.735120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.735621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.736095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.739368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.822741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.824355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.837177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1025/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:18.124322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.126081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:18.140835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da286b-dd90-498a-b40c-bb0bde87467e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.147708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.148638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.148739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.148963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] PATH_INFO: `/v3/projects/e09c39e301ad4cb2b7dd354345ab8d5a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.150102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09c39e301ad4cb2b7dd354345ab8d5a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.152001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.216018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09c39e301ad4cb2b7dd354345ab8d5a', 'request_id': 'req-3452d5d6-5688-43f3-917b-7ed7b633fb94'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.226071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09c39e301ad4cb2b7dd354345ab8d5a', 'request_id': 'req-3452d5d6-5688-43f3-917b-7ed7b633fb94'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.237532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09c39e301ad4cb2b7dd354345ab8d5a', 'request_id': 'req-3452d5d6-5688-43f3-917b-7ed7b633fb94'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.250615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] The token cache is being invalidate because project e09c39e301ad4cb2b7dd354345ab8d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:18.250905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.251567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.256694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1023/2048] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:18 2022] DELETE /identity/v3/projects/e09c39e301ad4cb2b7dd354345ab8d5a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:18.260349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.274508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d8f9-ccd1-4d05-a838-e7f80192cbda service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.350896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.351772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.352059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.352329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] PATH_INFO: `/v3/users/a9e80d2b4e9e43b7b1e98c3b7af82088` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.353571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e80d2b4e9e43b7b1e98c3b7af82088)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.355488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.418103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e80d2b4e9e43b7b1e98c3b7af82088'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.427326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e80d2b4e9e43b7b1e98c3b7af82088'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.437342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e80d2b4e9e43b7b1e98c3b7af82088'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.438047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e80d2b4e9e43b7b1e98c3b7af82088'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.465609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:18.469465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e80d2b4e9e43b7b1e98c3b7af82088'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.482569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1026/2049] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:18 2022] DELETE /identity/v3/users/a9e80d2b4e9e43b7b1e98c3b7af82088 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:18.792700 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.794748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:18.810677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d5d6-5688-43f3-917b-7ed7b633fb94 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.881556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.882201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.882481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.882761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] PATH_INFO: `/v3/projects/d973e12634da4b388cea66eaf24f9878` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.883916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d973e12634da4b388cea66eaf24f9878)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.885862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.938115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:18.940546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd973e12634da4b388cea66eaf24f9878', 'request_id': 'req-c4122a41-61b8-4436-b0ff-0582e04b013a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.950633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd973e12634da4b388cea66eaf24f9878', 'request_id': 'req-c4122a41-61b8-4436-b0ff-0582e04b013a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.957783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd973e12634da4b388cea66eaf24f9878', 'request_id': 'req-c4122a41-61b8-4436-b0ff-0582e04b013a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.969334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] The token cache is being invalidate because project d973e12634da4b388cea66eaf24f9878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:18.969732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.970454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:18.975409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1024/2050] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:18 2022] DELETE /identity/v3/projects/d973e12634da4b388cea66eaf24f9878 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:18.976359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.977857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:18.991382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6eb-31c7-4861-a582-39b891cb1f57 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.050266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.050909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.051200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.051476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] PATH_INFO: `/v3/users/a88c7eafd39945a9bd03cebab3fcf352` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.052690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88c7eafd39945a9bd03cebab3fcf352)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.054757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.108788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88c7eafd39945a9bd03cebab3fcf352'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.118088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88c7eafd39945a9bd03cebab3fcf352'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.125534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88c7eafd39945a9bd03cebab3fcf352'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.126199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88c7eafd39945a9bd03cebab3fcf352'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.139513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-98367eb3-ca25-4989-828d-282437721d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88c7eafd39945a9bd03cebab3fcf352'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.146263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1027/2051] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:18 2022] DELETE /identity/v3/users/a88c7eafd39945a9bd03cebab3fcf352 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:19.418140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.419683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:19.435400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4122a41-61b8-4436-b0ff-0582e04b013a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.504145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.504979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.505104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.505402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] PATH_INFO: `/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.508962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ebdc2810e641f4a8b6229ff259a6a9)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.510655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.562001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ebdc2810e641f4a8b6229ff259a6a9', 'request_id': 'req-a7fb17af-dcf5-491a-a375-8b74e281f95f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.570908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ebdc2810e641f4a8b6229ff259a6a9', 'request_id': 'req-a7fb17af-dcf5-491a-a375-8b74e281f95f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.577741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ebdc2810e641f4a8b6229ff259a6a9', 'request_id': 'req-a7fb17af-dcf5-491a-a375-8b74e281f95f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.590549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] The token cache is being invalidate because project a0ebdc2810e641f4a8b6229ff259a6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:19.590849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.591511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a7fb17af-dcf5-491a-a375-8b74e281f95f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:19.593369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1025/2052] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:19 2022] DELETE /identity/v3/projects/a0ebdc2810e641f4a8b6229ff259a6a9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:19.598489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.599002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.599002 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.623898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:19.624055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:19.700241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1028/2053] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:19.709733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.710068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.710360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.733648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:19.734925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:19.746720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1026/2054] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:19.755752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.769754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7522dc7-24b9-418d-8fc0-3684d42dcb50 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.774959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.775591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.775880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.776183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.776902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:19.777211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.778851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.799060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:19.807032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1029/2055] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:19.814106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.826354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76f091-5065-406d-bd70-da8121da02ce None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.844893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.845533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.845860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.846137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.847235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.849263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.889234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1027/2056] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:19 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:19.895558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.911350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308c7d6-750b-455b-9926-fc883cbb1ebf admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.989786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.990546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.990923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.991294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.992635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.994954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.023700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1030/2057] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:19 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:20.025078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:20.043046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761b3c3-eae6-401b-8b7b-d6eb59b46798 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:20.051450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:20.052065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.052350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.053109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.053306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:20.054925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.068662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1028/2058] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:20.070785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:20.082008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b293d8-3bfc-4fbb-96eb-a1ec72fdd839 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:20.092295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:20.092947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.093230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.093501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.094150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:20.095837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca47b7-cda8-4eac-9f01-eb9d545e3888 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.108283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1031/2059] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:20.113545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:20.129167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46585349-ab8b-4719-be30-38faf079965d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:20.134292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:20.134944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.135222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.135491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] PATH_INFO: `/v3/projects/928b73c7a8af4b49a2fd6d61abcf22d5/users/80ec12d0b78d4c38bd5f88335ac45119/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.151976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] RBAC: Authorizing `identity:create_grant(project_id=928b73c7a8af4b49a2fd6d61abcf22d5, user_id=80ec12d0b78d4c38bd5f88335ac45119, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:20.154761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.171112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1029/2060] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:20 2022] PUT /identity/v3/projects/928b73c7a8af4b49a2fd6d61abcf22d5/users/80ec12d0b78d4c38bd5f88335ac45119/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:20.183804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902651ed-ba59-41af-8745-c01e02e91b47 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.184089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902651ed-ba59-41af-8745-c01e02e91b47 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.184363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902651ed-ba59-41af-8745-c01e02e91b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.219814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-902651ed-ba59-41af-8745-c01e02e91b47 None None] MFA Rules not processed for user `80ec12d0b78d4c38bd5f88335ac45119`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:20.221113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-902651ed-ba59-41af-8745-c01e02e91b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:20.290025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1032/2061] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:20.304407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:20.306314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:20.310590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:20.397306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b2f4b-f287-4920-983f-13e2f48c425a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:20.401220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:20.401827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.402105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.402379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.411750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:20.413453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892646d-724c-4e31-a089-20ffeada17f5 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.433004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1030/2062] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:24.088895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cce90b-8f5b-41c8-93d9-9ce65f845c6a None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.090092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cce90b-8f5b-41c8-93d9-9ce65f845c6a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.090092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cce90b-8f5b-41c8-93d9-9ce65f845c6a None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.091234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1033/2063] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:35:24.115807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.116842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.117120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.142150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:24.145933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.272462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c22e3-d237-4283-9956-b4255f9b32ef None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.272858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c22e3-d237-4283-9956-b4255f9b32ef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.275526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c22e3-d237-4283-9956-b4255f9b32ef None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.275526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1034/2064] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:35:24.286087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4911-48ed-4aec-8a5c-e4c49a6a87d8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.286814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4911-48ed-4aec-8a5c-e4c49a6a87d8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.287154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4911-48ed-4aec-8a5c-e4c49a6a87d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.314864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1031/2065] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:35:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:24.315740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5a4a4911-48ed-4aec-8a5c-e4c49a6a87d8 None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:24.317190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5a4a4911-48ed-4aec-8a5c-e4c49a6a87d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.331692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1035/2066] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:35:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:24.355086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:24.357517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.367181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e757-9c41-4728-b1cd-ef23922b13d6 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.367472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e757-9c41-4728-b1cd-ef23922b13d6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.367759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7e757-9c41-4728-b1cd-ef23922b13d6 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.369214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1036/2067] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 29 21:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:35:24.375790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.376077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.376687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.379636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076b37f-1ad1-4b6c-a898-a880685d5abf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:24.385752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:24.386445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.386788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.387792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.400091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:24.402471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:24.409535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] MFA Rules not processed for user `0697bbbe44c74c538761737601ceeb10`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:24.410754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.420030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1037/2068] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 29 21:35:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:24.420799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1032/2069] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:24.422526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:24.423938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.427443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:24.441758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e18e1b-a556-4ecb-9334-440276ada810 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:24.449803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:24.449803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.449803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.449803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.461498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:24.463452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:24.473763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1033/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:24.495367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:24.503830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4d734-01a7-4390-a4b5-7ca736f22983 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:24.509121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:24.509765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.510138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.510312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] PATH_INFO: `/v3/users/b8ac60b941b04b2eb01281ba5f0e0b96` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.511457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ac60b941b04b2eb01281ba5f0e0b96)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:24.513988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:24.586367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ac60b941b04b2eb01281ba5f0e0b96'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:24.594625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ac60b941b04b2eb01281ba5f0e0b96'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:24.603952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ac60b941b04b2eb01281ba5f0e0b96'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:24.604793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ac60b941b04b2eb01281ba5f0e0b96'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:24.619155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ac60b941b04b2eb01281ba5f0e0b96'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:24.625774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1038/2071] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:24 2022] DELETE /identity/v3/users/b8ac60b941b04b2eb01281ba5f0e0b96 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:24.854414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:24.874043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505e6a0-43cf-48ce-abf2-c3b46fa3fcf8 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:24.958260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:24.967952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:24.968594 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:24.968897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:24.969176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] PATH_INFO: `/v3/projects/cb5424a2fd9d45c899b4ff8b7869f50e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:24.970325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5424a2fd9d45c899b4ff8b7869f50e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:24.972299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:24.975450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:24.977550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:24.995256 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3ba60-0443-415d-a72a-28e346539d02 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.001356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.002580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.003271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.003552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] PATH_INFO: `/v3/users/66df13267c5f4b14abb31f1bd88241d9` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.004765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66df13267c5f4b14abb31f1bd88241d9)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.006959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.029031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5424a2fd9d45c899b4ff8b7869f50e', 'request_id': 'req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.039782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5424a2fd9d45c899b4ff8b7869f50e', 'request_id': 'req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.048171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5424a2fd9d45c899b4ff8b7869f50e', 'request_id': 'req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.060842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] The token cache is being invalidate because project cb5424a2fd9d45c899b4ff8b7869f50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:25.061297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.062182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-6ae9c899-2ce8-47fd-8404-05bc0804ed2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.063273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66df13267c5f4b14abb31f1bd88241d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.064170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1034/2072] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:24 2022] DELETE /identity/v3/projects/cb5424a2fd9d45c899b4ff8b7869f50e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:25.070390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.070681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.071031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.075170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66df13267c5f4b14abb31f1bd88241d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.085140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66df13267c5f4b14abb31f1bd88241d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.086140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66df13267c5f4b14abb31f1bd88241d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.099031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:25.099721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a6ae3a0f-2d0e-4b47-bb4f-cf91bea5f393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66df13267c5f4b14abb31f1bd88241d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.100255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.106124 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1039/2073] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:24 2022] DELETE /identity/v3/users/66df13267c5f4b14abb31f1bd88241d9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:25.171722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1035/2074] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:25.179475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.179848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.180129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.201885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:25.203112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.252535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:25.280852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1040/2075] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:25.282731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.293744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290999f-af2b-4178-9a12-7aaeb1d3d946 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.303062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.303580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.307658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.308711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.309085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.309359 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.310248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:25.310692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.312842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.327072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2714e4f-e68b-4fc5-92d9-77fbaff16513 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.328630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1036/2076] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:25.333005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.334535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.335270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.335621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.335976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] PATH_INFO: `/v3/projects/7faae8d00a06427983b2944792166135` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.337356 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7faae8d00a06427983b2944792166135)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.339346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.346026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789647a-e90c-4a12-964e-0a14297f89bd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.351259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.352219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.352508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.352829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.353477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.355396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.397082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1037/2077] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:25 2022] POST /identity/v3/projects => generated 361 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:25.401982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.405289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7faae8d00a06427983b2944792166135', 'request_id': 'req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.417684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7faae8d00a06427983b2944792166135', 'request_id': 'req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.434024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7faae8d00a06427983b2944792166135', 'request_id': 'req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.451352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] The token cache is being invalidate because project 7faae8d00a06427983b2944792166135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:25.451711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.453850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.459152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1041/2078] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:25 2022] DELETE /identity/v3/projects/7faae8d00a06427983b2944792166135 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:25.465520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.469304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.484947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:25.513116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26107-daf9-489c-a424-3917b6dff32a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.529668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.529668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.529668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.529668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.530151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.531491 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.557766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1038/2079] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:25 2022] POST /identity/v3/users => generated 281 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:25.562924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.565487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.575485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d99f8e-ada1-4017-ba0b-e145fa9d6fb7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.582011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.582666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.582946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.583221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] PATH_INFO: `/v3/users/6a0c5afd54634878a168954cb2d5bcaf` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.584399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a0c5afd54634878a168954cb2d5bcaf)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.585350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf8a1a-e840-4700-8a04-75b07c7f2b6b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.586982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.591800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.592418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.592735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.593040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.593964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.595635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.610419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1039/2080] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:25.612527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.626014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12dbb0-b582-4285-b6a1-75e4bc45f73c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.631783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.632602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.632905 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.633475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.634150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.636147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.643437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a0c5afd54634878a168954cb2d5bcaf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.651671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1040/2081] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:25.652538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a0c5afd54634878a168954cb2d5bcaf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.652971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.667461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a0c5afd54634878a168954cb2d5bcaf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.667912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d619-e206-45a0-8493-525a54421e99 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.668454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a0c5afd54634878a168954cb2d5bcaf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.673556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.674215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.674493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.674834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] PATH_INFO: `/v3/projects/ad39b4e8eff94390856f3879af053975/users/c2793b0b4492469c9baa7ba7e653da31/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.678339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:25.685655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-39dc27ab-fd27-4fbd-b214-8a24612eb51a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a0c5afd54634878a168954cb2d5bcaf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:25.694044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1042/2082] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:25 2022] DELETE /identity/v3/users/6a0c5afd54634878a168954cb2d5bcaf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:25.694581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad39b4e8eff94390856f3879af053975, user_id=c2793b0b4492469c9baa7ba7e653da31, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.697519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.716161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1041/2083] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:25 2022] PUT /identity/v3/projects/ad39b4e8eff94390856f3879af053975/users/c2793b0b4492469c9baa7ba7e653da31/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:25.726436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.726755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.727031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.765155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] MFA Rules not processed for user `c2793b0b4492469c9baa7ba7e653da31`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:25.766411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.842061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1043/2084] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:25.862428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.862428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.898639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:25.900177 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:25.943698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aad75-963b-4e77-b049-8061cbf06da9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.947864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.948481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.948796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.949069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.958611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.960707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa10da-ea03-49b6-b944-b3114269970a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:25.970148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1042/2085] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:25.988654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9773b51-ce00-44b4-bf30-7a0c8733f09f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:25.995407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:25.996080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:25.996367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:25.996701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] PATH_INFO: `/v3/projects/bf2f032d6edd45349370a7d99dc4ee78` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:25.997848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2f032d6edd45349370a7d99dc4ee78)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:25.999780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.057007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2f032d6edd45349370a7d99dc4ee78', 'request_id': 'req-72b90e1d-6980-48c1-8a87-071d9bd04445'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:26.065288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2f032d6edd45349370a7d99dc4ee78', 'request_id': 'req-72b90e1d-6980-48c1-8a87-071d9bd04445'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:26.073616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2f032d6edd45349370a7d99dc4ee78', 'request_id': 'req-72b90e1d-6980-48c1-8a87-071d9bd04445'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:26.086703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] The token cache is being invalidate because project bf2f032d6edd45349370a7d99dc4ee78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:26.086993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:26.087667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-72b90e1d-6980-48c1-8a87-071d9bd04445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:26.089412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1044/2086] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:25 2022] DELETE /identity/v3/projects/bf2f032d6edd45349370a7d99dc4ee78 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:26.094332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.094850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.095190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.119231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:26.120644 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:26.196824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1043/2087] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:26.205344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.205886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.205886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.233185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:26.234444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:26.244844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1045/2088] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:26.248958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.266185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0ef70-2e29-4054-9f00-949e1d8afd7a None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.272773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.273539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.273900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.274211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.275051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:26.275519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.277958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.294827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1044/2089] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:26.300204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.305483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:26.317673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f075c6-3913-43f5-8ae9-21a7937d35aa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.330760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.331617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.332082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.332358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.333045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.335163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.370268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1046/2090] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:26 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:26.374477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.386763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb2a7f-bbbc-4a1e-91be-427d75d3eb69 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.412193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:26.451589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.452200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.452481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.452790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.453429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.455965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.481251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1045/2091] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:26 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:26.485397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.497119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa1695-9fec-40fc-a632-5c4e86e41037 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.502587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.503227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.503585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.503870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.504547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.506290 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.523811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1047/2092] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:26.527495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.537803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c3eb6-ff10-4d89-9526-312b7eb90382 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.545846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.546893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.547151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.547729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.548888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.550911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fddcf-7f9f-421c-8732-fd1660e6a052 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.564027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1046/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:26.569010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.580547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd352777-dd33-4122-8439-c76eeeccbf00 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.587554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.588279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.588403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.590410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] PATH_INFO: `/v3/projects/22bfa16047da4a6b8dac3021e5de85be/users/e547192628f24a208ee806dcc4960d70/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.613310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22bfa16047da4a6b8dac3021e5de85be, user_id=e547192628f24a208ee806dcc4960d70, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.616629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.649623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1048/2094] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:26 2022] PUT /identity/v3/projects/22bfa16047da4a6b8dac3021e5de85be/users/e547192628f24a208ee806dcc4960d70/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:26.658703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.659091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.659480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.702941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] MFA Rules not processed for user `e547192628f24a208ee806dcc4960d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:26.704181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:26.798299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1047/2095] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:26.810791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:26.812538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:26.877983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d3da4-ee13-4a3e-9659-7d37ce90efd2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:26.881644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:26.882265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:26.882547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:26.882823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:26.892625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:26.894016 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbead0f-7425-4fbd-a6fd-0dbb4a8e53e5 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:26.905633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1049/2096] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:35.812189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:35.888003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98995d6b-4f1b-4055-b0b6-db4ad7fe380b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:35.902927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:35.904039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:35.904039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:35.904222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] PATH_INFO: `/v3/users/e547192628f24a208ee806dcc4960d70` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:35.906345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e547192628f24a208ee806dcc4960d70)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:35.910230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:35.962969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:35.976043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e547192628f24a208ee806dcc4960d70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:35.988526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e547192628f24a208ee806dcc4960d70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:35.996370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e547192628f24a208ee806dcc4960d70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:35.997179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e547192628f24a208ee806dcc4960d70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:35.997808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e547192628f24a208ee806dcc4960d70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.015591 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1048/2097] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:35 2022] DELETE /identity/v3/users/e547192628f24a208ee806dcc4960d70 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:36.021292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.021769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.021901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.042304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:36.043857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.111419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1050/2098] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:36.121812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.123263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.191854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198dffb-6c9b-4976-bad9-12c588d772b9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.195950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.196573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.196873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.197139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.207588 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.209177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d945a-11b4-4234-91ac-794f0ad7bc83 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.219901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1049/2099] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:36.550207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.551695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.566755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31761dec-9a6e-4d55-9b7d-3a9b840b3a8b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.573205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.573863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.574140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.574414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] PATH_INFO: `/v3/projects/22bfa16047da4a6b8dac3021e5de85be` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.575562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=22bfa16047da4a6b8dac3021e5de85be)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.578690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.632192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:36.636737 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22bfa16047da4a6b8dac3021e5de85be', 'request_id': 'req-00d780a2-db87-4820-80e8-788eabb610ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.645903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22bfa16047da4a6b8dac3021e5de85be', 'request_id': 'req-00d780a2-db87-4820-80e8-788eabb610ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.654154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22bfa16047da4a6b8dac3021e5de85be', 'request_id': 'req-00d780a2-db87-4820-80e8-788eabb610ee'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.667175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] The token cache is being invalidate because project 22bfa16047da4a6b8dac3021e5de85be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:36.667467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.668133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00d780a2-db87-4820-80e8-788eabb610ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:36.670097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1051/2100] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:36 2022] DELETE /identity/v3/projects/22bfa16047da4a6b8dac3021e5de85be => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:36.677329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.677614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.677894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.699776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:36.701944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.775444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1050/2101] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:36.784689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.784986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.785247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.810319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:36.811744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.823112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1052/2102] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:36.827550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.842514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8358ed-dbf5-4710-b07d-b940c19dd953 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.847230 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.847838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.848806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.848806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.849178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:36.849391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.850968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.869143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1051/2103] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:35:36.874072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.886204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886f046-bd4f-4357-9678-93ec8164e3bd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.891107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.893335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.893858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.894582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.895603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.898515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.931455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1053/2104] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:36.935538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.947292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ef98-8f48-44f0-a308-bafbc4add7c9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.983675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.987190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.015692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:37.038525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.039146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.039422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.039694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.040342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.042421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.074387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1052/2105] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:37.077158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.090464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc383b3-faa0-4eaf-8280-1f430fb2d2e4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.090686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77469f26-d961-4bbf-ad43-d8001b9a50e1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.095780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.096500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.096868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.097203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] PATH_INFO: `/v3/users/c2793b0b4492469c9baa7ba7e653da31` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.097326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.098018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.098391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.098616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2793b0b4492469c9baa7ba7e653da31)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.098777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.099684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.100992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.102202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.120322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1053/2106] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:37.121143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.131447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7105-feed-49cd-961f-02ed9cd2095c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.137492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.138432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.138767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.139046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.139697 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.141365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.159407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1054/2107] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:37.163430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.166245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2793b0b4492469c9baa7ba7e653da31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.175475 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2793b0b4492469c9baa7ba7e653da31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.183532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2793b0b4492469c9baa7ba7e653da31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.184334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2793b0b4492469c9baa7ba7e653da31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.186363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bdd5-f46c-436a-9810-ea6f0aae6151 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.192085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.192811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.193122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.193398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] PATH_INFO: `/v3/projects/98d4703c6737454291724a5ad15c9cbb/users/d882c87d5de7497b822ec7b1798d2008/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.200064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6407ba9b-3df4-4b25-b6e9-3c816c4d8cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2793b0b4492469c9baa7ba7e653da31'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.214493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1054/2108] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:36 2022] DELETE /identity/v3/users/c2793b0b4492469c9baa7ba7e653da31 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:37.214621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d4703c6737454291724a5ad15c9cbb, user_id=d882c87d5de7497b822ec7b1798d2008, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.215714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.215988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.216173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158aea6-71e7-49f7-bb0c-a1225986a51c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.216275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.233105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1055/2109] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:37 2022] PUT /identity/v3/projects/98d4703c6737454291724a5ad15c9cbb/users/d882c87d5de7497b822ec7b1798d2008/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:37.239315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.239315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.239315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.241998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:37.243415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.266339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:37.267611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.283382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:37.337611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1055/2110] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:37.350823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.352359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.363664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1056/2111] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:37.373582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.375128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.428792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ab74c-142c-49f7-82a1-86f917f7a2cb None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.432807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.433556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.433942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.434321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.448797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.450709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed305b8-f973-48c7-8918-137ef12fbe5b None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.450840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.454685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.455388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.455743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.456205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.462974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1056/2112] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:37.466837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.468221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7b139-4823-486c-8764-014889c8e6ab service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.480208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1057/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:37.870292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:37.872714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:37.905333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90654f13-e440-45cb-acb9-757df85ed50a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:37.913813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:37.916795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:37.917101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:37.917384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] PATH_INFO: `/v3/projects/ad39b4e8eff94390856f3879af053975` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:37.918602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad39b4e8eff94390856f3879af053975)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:37.920562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:37.986832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad39b4e8eff94390856f3879af053975', 'request_id': 'req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:37.992749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad39b4e8eff94390856f3879af053975', 'request_id': 'req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:38.003824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad39b4e8eff94390856f3879af053975', 'request_id': 'req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:38.019831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] The token cache is being invalidate because project ad39b4e8eff94390856f3879af053975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:38.020334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:38.021051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-be5d7d3b-cff4-4072-a3ee-06c4acb4a61c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:38.026051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1057/2114] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:37 2022] DELETE /identity/v3/projects/ad39b4e8eff94390856f3879af053975 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:38.027774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.028243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.028338 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.058939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:38.060274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:38.087341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:38.125822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1058/2115] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:38.135107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.135618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.135703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.158749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:38.160029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:38.170078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1058/2116] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:38.174543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.184688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0e5f-068e-4fad-8ebe-045c35962b79 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.189598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.190220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.190508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.191105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.191442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:38.191739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.193460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.209803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1059/2117] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:38.216647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.232614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca0b26-f538-4607-ae2e-e3c9c313583f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.240663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.241695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.242130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.242525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.243392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.246264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.308142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1059/2118] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/projects => generated 357 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:38.314567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.332188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d79a6-ea24-499d-b082-b7d09f5621f1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.409093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.410255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.410870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.412189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.412691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.416657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.449743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1060/2119] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/users => generated 279 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:38.454456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.471169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae630d39-74cb-4b09-aeb9-be2054ef184a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.481902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.489289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.493419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.494255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.495800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.498070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.513554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1060/2120] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:38.517446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.529164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc3b5f-c645-4986-a3a7-90fbe1ec0e42 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.535007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.536390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.536390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.536773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.537379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.539088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5076-8306-4a88-bf28-f3c2f683449f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.552629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1061/2121] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:38.557566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.572221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75609c-54c5-4948-883f-5e8065c333ee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.578222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.578703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.579109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.579542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] PATH_INFO: `/v3/projects/6968b2df087d4e7986f854244533d10a/users/5381f8f6a5cd482ea8ec8bd7865d1598/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.588835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:38.611231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6968b2df087d4e7986f854244533d10a, user_id=5381f8f6a5cd482ea8ec8bd7865d1598, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.614886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.643686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1061/2122] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:38 2022] PUT /identity/v3/projects/6968b2df087d4e7986f854244533d10a/users/5381f8f6a5cd482ea8ec8bd7865d1598/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:38.658604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.658955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.659256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.689586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] MFA Rules not processed for user `5381f8f6a5cd482ea8ec8bd7865d1598`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:38.690801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:38.779823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1062/2123] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:38.781423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.856705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df3ac8-459a-4fb8-88f3-c261a4b3510c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.866289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:38.867159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:38.867604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:38.868017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:38.869056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:38.871264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:38.912163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1062/2124] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/projects => generated 357 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:38.913334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:38.924763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d3608-623d-438d-8f39-6744b1c473c2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:38.951927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:39.012143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.012729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.013015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.013300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.014829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.016511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.054298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1063/2125] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:38 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:39.059292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.092043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf8c2a-80d1-485c-abac-d417d4ae9c5f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.101576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.102129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.102341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.102576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.103652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.113819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.129833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1063/2126] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:39.134086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.154522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3f225-3c39-4d81-adcb-ca7e0e6e91d7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.160138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.161082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.161461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.161854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] PATH_INFO: `/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb/users/61f0d1653dcf4c6b81bdc151b5ee8b88/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.183980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cd5ffb5ffe49f58a8edf8ec55f9abb, user_id=61f0d1653dcf4c6b81bdc151b5ee8b88, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.186689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.208216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1064/2127] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:39 2022] PUT /identity/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb/users/61f0d1653dcf4c6b81bdc151b5ee8b88/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:39.209554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.221113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9319738-9b02-4021-91a9-50de35fd93d2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.320320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.320911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.321905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.321905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.322547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.325684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83935d-1927-47a2-98b0-914ca48c87d0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.350831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1064/2128] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:39.355062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.370233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb604309-fe42-4179-aa9f-85fce97747bb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.380282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.381057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.381375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.383043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] PATH_INFO: `/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb/users/61f0d1653dcf4c6b81bdc151b5ee8b88/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.385269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cd5ffb5ffe49f58a8edf8ec55f9abb, user_id=61f0d1653dcf4c6b81bdc151b5ee8b88, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.388043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.401037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1065/2129] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:39 2022] PUT /identity/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb/users/61f0d1653dcf4c6b81bdc151b5ee8b88/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:39.415944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.416286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.416569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.447436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] MFA Rules not processed for user `61f0d1653dcf4c6b81bdc151b5ee8b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:39.448694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:39.497302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:39.525672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1065/2130] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:39.541854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.544173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:39.611042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958b75a-f9af-4ec9-be7f-5b6b01b5798f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.615978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.616694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.616923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.617197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.628059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.629569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58b2bb-7ff5-4627-a4af-85f0d534fb56 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.640408 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1066/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:39.723589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:39.734330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102844e-6f29-473e-8cb5-e12e693fe63d None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:39.738340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:39.739098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:39.739513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:39.739513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:39.805887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:39.807332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:39.817822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1066/2132] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:42.239013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.239523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.239606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.275278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] MFA Rules not processed for user `d882c87d5de7497b822ec7b1798d2008`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:42.276619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:42.371947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1067/2133] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:42.377678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:42.379224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:42.463059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79046312-9368-4d6d-99d2-98e6d85e18f8 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:42.468366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:42.469622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.470732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.471191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.472049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:42.474325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:42.512642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1067/2134] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:42 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:42.515245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:42.531549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65093e33-78b6-488b-a21a-680b858be9a2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:42.538942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:42.614820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:42.616221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.616713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.616998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.617671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:42.620337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:42.656611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1068/2135] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:42 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:42.658137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:42.680965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94b0bf-ec48-427a-bc20-d422c45521c7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:42.695998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:42.696661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.696945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.697222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.697881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:42.699581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:42.740842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1068/2136] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:42.745044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:42.778839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9716a4-0f46-4bb2-9872-f3f1209f7a34 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:42.785618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:42.786328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.786725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.787035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] PATH_INFO: `/v3/projects/2c25a88a2c3541c7a2928901758bda71/users/4100e961720a41d7b46ea0cd5b5f9a45/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.833557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c25a88a2c3541c7a2928901758bda71, user_id=4100e961720a41d7b46ea0cd5b5f9a45, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:42.836476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:42.869249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1069/2137] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:42 2022] PUT /identity/v3/projects/2c25a88a2c3541c7a2928901758bda71/users/4100e961720a41d7b46ea0cd5b5f9a45/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:42.870955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:42.882293 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf497267-cf7c-402f-b9e9-16c92d2a91f5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:42.966170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:42.978051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:42.979289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:42.979289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:42.979512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:42.980229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:42.982091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7afa-51b1-4dca-b8d3-ae208cd43676 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:42.993955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1069/2138] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:42.997510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:43.011209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e6087-be74-438f-ab9d-fd20776cdb9f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:43.022126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:43.022879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:43.023154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:43.023431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] PATH_INFO: `/v3/projects/2c25a88a2c3541c7a2928901758bda71/users/4100e961720a41d7b46ea0cd5b5f9a45/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:43.025422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c25a88a2c3541c7a2928901758bda71, user_id=4100e961720a41d7b46ea0cd5b5f9a45, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:43.028117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:43.040730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1070/2139] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:42 2022] PUT /identity/v3/projects/2c25a88a2c3541c7a2928901758bda71/users/4100e961720a41d7b46ea0cd5b5f9a45/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:47.887444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.888037 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.888740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.925423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] MFA Rules not processed for user `4100e961720a41d7b46ea0cd5b5f9a45`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:47.926854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:48.014497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1070/2140] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:48.036673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:48.039575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:48.145667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515adb3-13b8-4559-8289-4003050436df admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:48.157041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:48.158447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:48.158835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:48.159070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:48.221305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:48.222991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b0311-e9aa-4bb9-849c-0f4608c4f05f service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:48.226789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:48.235461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1071/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:58.007773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=39 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:58.236186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:00.154695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:00.156615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:00.249794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6c140-9f7a-4469-a0de-4aa44e6731d4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:00.260717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:00.261757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.262134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.262573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] PATH_INFO: `/v3/users/80ec12d0b78d4c38bd5f88335ac45119` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.264656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ec12d0b78d4c38bd5f88335ac45119)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:00.267101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:00.352794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ec12d0b78d4c38bd5f88335ac45119'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.362371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ec12d0b78d4c38bd5f88335ac45119'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.371830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ec12d0b78d4c38bd5f88335ac45119'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.372531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ec12d0b78d4c38bd5f88335ac45119'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.385886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ec12d0b78d4c38bd5f88335ac45119'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.393341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1071/2142] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:00 2022] DELETE /identity/v3/users/80ec12d0b78d4c38bd5f88335ac45119 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:00.397633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.398210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.398397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.418382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:00.419669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:00.488984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1072/2143] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:00.499321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:00.501570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:00.563932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e934e-2984-4b54-89a4-f18db4686444 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:00.567523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:00.568138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.568416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.568722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.578150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:00.579567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90d754-7b11-4fdd-b618-b5f89b1847fa service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:00.590189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1072/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:00.865157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:00.866708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:00.879017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318a31-e9c7-4cd6-a078-174f8bf5de09 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:00.883980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:00.884658 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.884943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.885235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] PATH_INFO: `/v3/projects/928b73c7a8af4b49a2fd6d61abcf22d5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.886354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928b73c7a8af4b49a2fd6d61abcf22d5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:00.888278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:00.944932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928b73c7a8af4b49a2fd6d61abcf22d5', 'request_id': 'req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.953164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928b73c7a8af4b49a2fd6d61abcf22d5', 'request_id': 'req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.961953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928b73c7a8af4b49a2fd6d61abcf22d5', 'request_id': 'req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.975726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] The token cache is being invalidate because project 928b73c7a8af4b49a2fd6d61abcf22d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:00.976014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.976705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0cd4768e-c1ef-4cff-82b9-90f19c2aad32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.978225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1073/2145] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:00 2022] DELETE /identity/v3/projects/928b73c7a8af4b49a2fd6d61abcf22d5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:00.983062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.983350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.983650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.003794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:01.004996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.024635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:01.078406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1073/2146] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:01.087970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.088266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.088548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.113493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:01.115369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.124939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1074/2147] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:01.129353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.140717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b3f60-be9c-4736-a449-b912ba7443ff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.148775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.149671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.149955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.150261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.150950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:01.151255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.153255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.171995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1074/2148] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:36:01.174331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.184479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb270d-be1b-49be-9ed4-2312ede0ce6e None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.190496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.191158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.191487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.191760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.192390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.194328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.227004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1075/2149] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:01 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:01.228662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.244024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922bc02-15b5-4437-88c0-2abd6edb799f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.269649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.272986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.312993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.314193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.314470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.314740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.315370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.317458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.341971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1075/2150] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:01 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:01.342185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:01.342995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.348256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256046c-b89d-40f8-a5b4-67360a7ffadd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.355219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cef3b-44ad-4624-be5a-072a939475df admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.363040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.363766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.364045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.364314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] PATH_INFO: `/v3/users/5381f8f6a5cd482ea8ec8bd7865d1598` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.365577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5381f8f6a5cd482ea8ec8bd7865d1598)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.366559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.368433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.368433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.368649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.369963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.371003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.373435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.389008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1076/2151] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:01.394058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.407685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6fc6-8926-427c-b310-c8bc791506de admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.414264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.414995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.415218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.415491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.416453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.419030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.436172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5381f8f6a5cd482ea8ec8bd7865d1598'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.436714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1077/2152] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:01.441286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.452336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22ec3-5c5f-4910-9db6-1ad30e0978fe admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.457870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.458034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5381f8f6a5cd482ea8ec8bd7865d1598'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.458262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.458546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.458828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] PATH_INFO: `/v3/projects/84bb44e395224371870b96d7a3fdf152/users/6f85d78501d4486dacd66dea638baf23/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.471432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5381f8f6a5cd482ea8ec8bd7865d1598'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.472782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5381f8f6a5cd482ea8ec8bd7865d1598'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.473633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28476608-4f50-489e-bf55-1dbe9bbf833c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5381f8f6a5cd482ea8ec8bd7865d1598'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.475515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=84bb44e395224371870b96d7a3fdf152, user_id=6f85d78501d4486dacd66dea638baf23, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.478248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03972f27-8f61-4a76-b9cc-e17c744524bf admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.494659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1078/2153] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:01 2022] PUT /identity/v3/projects/84bb44e395224371870b96d7a3fdf152/users/6f85d78501d4486dacd66dea638baf23/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:01.495124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1076/2154] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:01 2022] DELETE /identity/v3/users/5381f8f6a5cd482ea8ec8bd7865d1598 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:01.504735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.505028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.505301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.506424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.506713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.507056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.525568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:01.526825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.540716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] MFA Rules not processed for user `6f85d78501d4486dacd66dea638baf23`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:01.541662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.592497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:01.609151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1079/2155] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:01.621657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.623050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.626517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1077/2156] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:01.638215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.639825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.700829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294224fd-9fb0-4d9f-81b3-30e54fb2704f None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.705425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.707643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.707929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.708207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.722976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec4fd8-46de-46f2-80e0-ad09f39dd148 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.724902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.726751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.727444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.727967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.728310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.728657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.739602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1080/2157] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:01.739996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.741622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.758145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1078/2158] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:02.102551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:02.102551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:02.115651 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e175a-6c75-42ca-af71-73e750debfaf service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:02.124490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:02.125170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.126099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.126360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] PATH_INFO: `/v3/projects/6968b2df087d4e7986f854244533d10a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.128098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6968b2df087d4e7986f854244533d10a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:02.130067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:02.195784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6968b2df087d4e7986f854244533d10a', 'request_id': 'req-058856af-7aba-420d-b2db-ed0895efc835'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.201615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6968b2df087d4e7986f854244533d10a', 'request_id': 'req-058856af-7aba-420d-b2db-ed0895efc835'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.209720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6968b2df087d4e7986f854244533d10a', 'request_id': 'req-058856af-7aba-420d-b2db-ed0895efc835'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.221630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] The token cache is being invalidate because project 6968b2df087d4e7986f854244533d10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:02.221918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.222578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.224088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1081/2159] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:02 2022] DELETE /identity/v3/projects/6968b2df087d4e7986f854244533d10a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:02.228092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:02.239876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8401997-fd8a-40db-a221-6586da0243e7 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:02.243539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:02.299755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:02.300374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.300657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.300931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] PATH_INFO: `/v3/users/61f0d1653dcf4c6b81bdc151b5ee8b88` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.302025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f0d1653dcf4c6b81bdc151b5ee8b88)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:02.303993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:02.350894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f0d1653dcf4c6b81bdc151b5ee8b88'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.363047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f0d1653dcf4c6b81bdc151b5ee8b88'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.370195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f0d1653dcf4c6b81bdc151b5ee8b88'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.370871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f0d1653dcf4c6b81bdc151b5ee8b88'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.371470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82ecfae3-96ee-4e44-9a50-426b11db375a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f0d1653dcf4c6b81bdc151b5ee8b88'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.384685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1079/2160] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:02 2022] DELETE /identity/v3/users/61f0d1653dcf4c6b81bdc151b5ee8b88 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:02.679561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:02.681032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:02.702780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058856af-7aba-420d-b2db-ed0895efc835 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:02.761689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:02.762363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.762641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.762915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] PATH_INFO: `/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.764262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cd5ffb5ffe49f58a8edf8ec55f9abb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:02.766759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:02.794862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:02.821750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cd5ffb5ffe49f58a8edf8ec55f9abb', 'request_id': 'req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.827139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cd5ffb5ffe49f58a8edf8ec55f9abb', 'request_id': 'req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.834295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cd5ffb5ffe49f58a8edf8ec55f9abb', 'request_id': 'req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.845239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] The token cache is being invalidate because project d3cd5ffb5ffe49f58a8edf8ec55f9abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:02.845529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.846150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3ba7e90c-9868-4ccf-8a70-13c56e2b381a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.847627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1082/2161] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:02 2022] DELETE /identity/v3/projects/d3cd5ffb5ffe49f58a8edf8ec55f9abb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:02.853187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.853918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.854133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.878396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:02.879613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:02.950793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1080/2162] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:02.958932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.959219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.959517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.979219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:02.980441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:02.992809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1083/2163] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:02.994517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.005152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a326df-d096-412e-a6ab-f5ac90f5cb63 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.009600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.010195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.010468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.010758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.011393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:03.011686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.013355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.030695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1081/2164] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:36:03.032380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.042870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cdf62-1403-4c5a-9992-3bc69619a14a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.048220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.048881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.049165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.049467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.050101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.051998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.081797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1084/2165] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:03 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:03.087232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.101788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb997-0ce7-4fb2-a0d8-364e3f1ddcf8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.111264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:03.165918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.166629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.166910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.167185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.167845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.170048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.197563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1082/2166] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:03 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:03.198707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.209865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f9b05-c1e5-484b-a138-fb6c7fcfd876 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.223533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.224736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.224736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.224938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.225464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.228348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.245218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1085/2167] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:03.249563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.262920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8277188-87f7-41a1-8167-9ec026169a66 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.270435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.271061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.271347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.271981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.272279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.273971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f522023-2d7b-46d0-9ede-8cc7653e764b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.292813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1083/2168] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:03.293543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.303728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43c20-1405-4de5-b7d4-8b537672c096 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.308959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.309617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.310211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.310211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] PATH_INFO: `/v3/projects/860c6b0cbf5946dfa0384b8055067153/users/81cf90489d6b41f8809b1a6ea29b864c/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.326164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860c6b0cbf5946dfa0384b8055067153, user_id=81cf90489d6b41f8809b1a6ea29b864c, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.329486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.348449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1086/2169] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:03 2022] PUT /identity/v3/projects/860c6b0cbf5946dfa0384b8055067153/users/81cf90489d6b41f8809b1a6ea29b864c/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:03.364527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.364527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.364527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.405886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] MFA Rules not processed for user `81cf90489d6b41f8809b1a6ea29b864c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:03.407182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:03.521232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1084/2170] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:03.522112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.596363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698bdbb-48fa-4aa3-9480-13d1e2cc8597 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.602109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.602728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.603032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.603303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.604011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.606451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.638543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1087/2171] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:03 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:03.642776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.654966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fd5df-d079-47e8-a657-2a5a36e09af5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.723578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.724812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.725097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.725390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.726240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.728303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.754749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1085/2172] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:03 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:03.756410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.767512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca6bb-0764-4628-855c-5035ff87390a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.774738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.774738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.774738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.774738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.775642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.777342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.788464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1088/2173] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:03.792898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.803478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc5491-1324-49b9-808a-d22c9af6d0c0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.808571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.809525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.809868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.810171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] PATH_INFO: `/v3/projects/eecb589ec5944c879fa3981c36790e3c/users/d72e73a7be2f4789953148082c09e2d0/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.820443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:03.829404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecb589ec5944c879fa3981c36790e3c, user_id=d72e73a7be2f4789953148082c09e2d0, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.832441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.847124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1086/2174] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:03 2022] PUT /identity/v3/projects/eecb589ec5944c879fa3981c36790e3c/users/d72e73a7be2f4789953148082c09e2d0/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:03.851114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.862807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d8c77-5ad4-4ccf-80ea-25787cc06d46 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.894270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:03.936712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.937320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.937599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.938402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.939034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.940672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5f133-71c5-486b-9249-ab464f11e339 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.950988 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1089/2175] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:03.955714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.967574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3676f-0953-47a1-9470-211412b63798 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.981524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.982197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.982473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.982745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] PATH_INFO: `/v3/projects/eecb589ec5944c879fa3981c36790e3c/users/d72e73a7be2f4789953148082c09e2d0/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.984895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecb589ec5944c879fa3981c36790e3c, user_id=d72e73a7be2f4789953148082c09e2d0, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.987831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:04.006426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1087/2176] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:03 2022] PUT /identity/v3/projects/eecb589ec5944c879fa3981c36790e3c/users/d72e73a7be2f4789953148082c09e2d0/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:04.012606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.012964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.013246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.046168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] MFA Rules not processed for user `d72e73a7be2f4789953148082c09e2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:04.049727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.127586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1090/2177] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:04.139076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:04.140957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.213215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283c339-f4d3-458a-b90f-0c53939f9252 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:04.217273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:04.217912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.218191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.218460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.296634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:04.298049 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26635394-5483-4a59-87cb-c53bd978c8de service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:04.310714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1088/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:04.524236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:04.526676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.607897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65a5a7-4afc-4cfb-b34c-129c6d3aff77 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:04.614635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:04.615265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.615554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.615913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] PATH_INFO: `/v3/users/6f85d78501d4486dacd66dea638baf23` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.617473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f85d78501d4486dacd66dea638baf23)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:04.619746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:04.692108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:04.699320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f85d78501d4486dacd66dea638baf23'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.703718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.704002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.704281 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.711741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f85d78501d4486dacd66dea638baf23'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.725545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f85d78501d4486dacd66dea638baf23'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.728014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f85d78501d4486dacd66dea638baf23'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.734042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] MFA Rules not processed for user `5344cf7b391d44beac2c602ac3e2e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:04.735869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.744296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-29330bb0-8934-4fbf-aa96-3515c855dbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f85d78501d4486dacd66dea638baf23'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.752654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1091/2179] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:04 2022] DELETE /identity/v3/users/6f85d78501d4486dacd66dea638baf23 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:04.757118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.757606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.757892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.780039 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:04.782001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.811444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1089/2180] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:04.857302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1092/2181] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:04.865336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:04.866842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.907377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:04.936156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a3c4-9586-4d89-8170-001a8b39b0a4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:04.987566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:04.988344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.988640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.988921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.998788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.000246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.012638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1090/2182] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:05.472413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.473883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:05.500302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e8af-678e-4a7e-b6c8-416cd32ae6fe None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.508280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.508956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.509236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.509505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] PATH_INFO: `/v3/projects/84bb44e395224371870b96d7a3fdf152` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.510687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84bb44e395224371870b96d7a3fdf152)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.512614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.564331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84bb44e395224371870b96d7a3fdf152', 'request_id': 'req-1267a24a-e2c5-4e4a-a987-8226fb418a1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.570736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84bb44e395224371870b96d7a3fdf152', 'request_id': 'req-1267a24a-e2c5-4e4a-a987-8226fb418a1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.580519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84bb44e395224371870b96d7a3fdf152', 'request_id': 'req-1267a24a-e2c5-4e4a-a987-8226fb418a1f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.594933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] The token cache is being invalidate because project 84bb44e395224371870b96d7a3fdf152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:05.596832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.597879 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1267a24a-e2c5-4e4a-a987-8226fb418a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.599574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1093/2183] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:05 2022] DELETE /identity/v3/projects/84bb44e395224371870b96d7a3fdf152 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:10.135346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:10.136804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:10.204548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d603ee1-fd4a-410a-a0dd-bc180fc6de88 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:10.255146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:10.255810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:10.256087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:10.256357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:10.258248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:10.324150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:10.326207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a93747-17ec-4abf-a29e-57706d01549e service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:10.331332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:10.339566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1091/2184] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:12.564102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.564644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.564742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.584377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] MFA Rules not processed for user `89b272427c6444b49293e5046c686ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:12.585784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:12.627034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:12.642941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1094/2185] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:16.839808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb45a7-c586-4207-bec0-8f0d64595115 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:16.840329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb45a7-c586-4207-bec0-8f0d64595115 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:16.840440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb45a7-c586-4207-bec0-8f0d64595115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:16.861453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0afb45a7-c586-4207-bec0-8f0d64595115 None None] MFA Rules not processed for user `d882c87d5de7497b822ec7b1798d2008`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:16.862685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0afb45a7-c586-4207-bec0-8f0d64595115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:16.917650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1092/2186] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:22.637330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:26.910938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=26 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:35.946541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:35.948796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.050638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908b8cb-46fa-49df-b465-41530aa51f65 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:36.056938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:36.057684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:36.059162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:36.060200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] PATH_INFO: `/v3/users/81cf90489d6b41f8809b1a6ea29b864c` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:36.062291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81cf90489d6b41f8809b1a6ea29b864c)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:36.064863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:36.131304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81cf90489d6b41f8809b1a6ea29b864c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.138496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81cf90489d6b41f8809b1a6ea29b864c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.147197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81cf90489d6b41f8809b1a6ea29b864c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.147888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81cf90489d6b41f8809b1a6ea29b864c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.163404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81cf90489d6b41f8809b1a6ea29b864c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.169726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1095/2187] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:35 2022] DELETE /identity/v3/users/81cf90489d6b41f8809b1a6ea29b864c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:36.175308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:36.175768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:36.175861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:36.200178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:36.201613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.305613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1093/2188] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:36.316755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:36.318226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.423140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59eebe-745a-4c34-be6d-428b5d45b786 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:36.426849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:36.427611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:36.427893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:36.428163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:36.438593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:36.440548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:36.452533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1096/2189] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:36.794636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:36.796911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.818029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b765d6-464f-46f2-8bc9-bd1aeb2a1388 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:36.826710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:36.827430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:36.827711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:36.827987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] PATH_INFO: `/v3/projects/860c6b0cbf5946dfa0384b8055067153` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:36.829197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=860c6b0cbf5946dfa0384b8055067153)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:36.831131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:36.923097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860c6b0cbf5946dfa0384b8055067153', 'request_id': 'req-491b18a9-94ad-4629-9597-17d60d7e1efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.933948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860c6b0cbf5946dfa0384b8055067153', 'request_id': 'req-491b18a9-94ad-4629-9597-17d60d7e1efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.945369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860c6b0cbf5946dfa0384b8055067153', 'request_id': 'req-491b18a9-94ad-4629-9597-17d60d7e1efd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.959776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] The token cache is being invalidate because project 860c6b0cbf5946dfa0384b8055067153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:36.960095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.960788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:36.962886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1094/2190] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:36 2022] DELETE /identity/v3/projects/860c6b0cbf5946dfa0384b8055067153 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:36.966769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:36.977942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a784e6-b8ac-46d6-b736-25652085deb3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:37.002607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:37.053868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:37.054601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.054955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.055306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] PATH_INFO: `/v3/users/d72e73a7be2f4789953148082c09e2d0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.057326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72e73a7be2f4789953148082c09e2d0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:37.059407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:37.144604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72e73a7be2f4789953148082c09e2d0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.153757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72e73a7be2f4789953148082c09e2d0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.163894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72e73a7be2f4789953148082c09e2d0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.165178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72e73a7be2f4789953148082c09e2d0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.184558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0d68c176-f910-49fd-8b35-2f8d5a199358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72e73a7be2f4789953148082c09e2d0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.193283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1097/2191] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:36 2022] DELETE /identity/v3/users/d72e73a7be2f4789953148082c09e2d0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:37.527721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:37.529229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:37.556912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b18a9-94ad-4629-9597-17d60d7e1efd admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:37.633780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:37.634472 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.634754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.635033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] PATH_INFO: `/v3/projects/eecb589ec5944c879fa3981c36790e3c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.636251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecb589ec5944c879fa3981c36790e3c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:37.638312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:37.680038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:37.704999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecb589ec5944c879fa3981c36790e3c', 'request_id': 'req-fe1fd23d-3981-44bd-ab87-11ccb0902e57'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.717303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecb589ec5944c879fa3981c36790e3c', 'request_id': 'req-fe1fd23d-3981-44bd-ab87-11ccb0902e57'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.724879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecb589ec5944c879fa3981c36790e3c', 'request_id': 'req-fe1fd23d-3981-44bd-ab87-11ccb0902e57'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.739730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] The token cache is being invalidate because project eecb589ec5944c879fa3981c36790e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:37.740023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.740961 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-fe1fd23d-3981-44bd-ab87-11ccb0902e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:37.743522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1095/2192] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:37 2022] DELETE /identity/v3/projects/eecb589ec5944c879fa3981c36790e3c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:37.748298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.748618 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.748909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.777901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:37.779236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:37.884829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1098/2193] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:37.898019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.898309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.898583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.919876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:37.921162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:37.931100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1096/2194] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:37.938004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:37.948572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02fe5-0a7c-4447-8bfb-55eef81355ff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:37.954433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:37.955060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.955336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.955604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.956263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:37.956559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:37.958208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:37.974664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1099/2195] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:37.977055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:37.986447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954695d-d704-4471-ac1d-928c06fcbfa9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:37.991116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:37.991721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:37.991998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:37.992265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:37.992943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:37.994841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.031742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1097/2196] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:37 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:38.037772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.050342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e98b6-bb0b-45da-8837-66c0b76c80d6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.083120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:38.118971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.119245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.119462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.119741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.120383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.122393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.146003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1100/2197] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:38 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:38.150331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.161817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bd592-96ab-4220-9bf7-3b257ceb6341 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.166657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.167927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.168208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.168474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.169191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.170806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.182157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1098/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:38.186053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.196961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126dbc-6d63-4291-a533-e1da09b148ae admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.201708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.202315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.202594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.202866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.203508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.205135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e618e8-7fb0-47c1-8df9-ccbdbad1c131 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.216924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1101/2199] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:38.221849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.231899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b24cf-ad77-42f4-b37c-d6805cf07d0f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.237614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.238353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.238728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.239064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] PATH_INFO: `/v3/projects/b6d6514de9024d84bf73a0c0de580779/users/695dfd31f7da435e8c6544b29a5298e1/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.256743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d6514de9024d84bf73a0c0de580779, user_id=695dfd31f7da435e8c6544b29a5298e1, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.259440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.275766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1099/2200] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:38 2022] PUT /identity/v3/projects/b6d6514de9024d84bf73a0c0de580779/users/695dfd31f7da435e8c6544b29a5298e1/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:38.287123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.287429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.287663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.318222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] MFA Rules not processed for user `695dfd31f7da435e8c6544b29a5298e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:38.319452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:38.392404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1102/2201] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:38.393790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.474459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49567e-1170-45f1-a59d-5a86a0f7bef4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.480263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.480994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.481292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.481475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.482146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.485158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.517009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1100/2202] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:38 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:36:38.519583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.531350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ebc46e-e98b-4438-878b-03ff6be5bbcd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.598580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.599260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.599610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.599950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.600685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.602720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.628520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1103/2203] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:38 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:38.633711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.645943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39885748-322a-45cc-875a-be95a2167c11 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.651609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.652256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.652535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.652837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.653483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.655105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.670007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1101/2204] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:38.671815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.683064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451d642-407b-4d83-8ae4-4f103993fd04 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.687891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.688551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.688864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.689141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] PATH_INFO: `/v3/projects/99ff7a55ff27448f986518ff5499b4cc/users/ced992120e514528a9288b6c1a42f59a/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.704964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ff7a55ff27448f986518ff5499b4cc, user_id=ced992120e514528a9288b6c1a42f59a, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.707868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.722847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1104/2205] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:38 2022] PUT /identity/v3/projects/99ff7a55ff27448f986518ff5499b4cc/users/ced992120e514528a9288b6c1a42f59a/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:38.726657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.739285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0e083-1896-4e70-8111-10328d454764 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.761184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:38.805797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.806452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.806724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.806994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.807655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.809314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61090f7b-0bc7-435e-8df6-302f2265b811 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.820667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1102/2206] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:38.824720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:38.831111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:38.838292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc4ca2-6176-4f28-a6b4-df3a1f3cfeea admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:38.844241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:38.844949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.845227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.845495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] PATH_INFO: `/v3/projects/99ff7a55ff27448f986518ff5499b4cc/users/ced992120e514528a9288b6c1a42f59a/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.847442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ff7a55ff27448f986518ff5499b4cc, user_id=ced992120e514528a9288b6c1a42f59a, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:38.850268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:38.868515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1105/2207] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:38 2022] PUT /identity/v3/projects/99ff7a55ff27448f986518ff5499b4cc/users/ced992120e514528a9288b6c1a42f59a/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:38.879538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:38.880019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:38.880305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:38.916468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] MFA Rules not processed for user `ced992120e514528a9288b6c1a42f59a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:38.918167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:39.003158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1103/2208] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:39.016858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:39.018904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:39.086837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4b1da-8350-4ca9-a44c-49fe68783158 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:39.091076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:39.091699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:39.091978 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:39.092396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:39.103107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:39.104848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:39.120968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1106/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:39.171756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:39.173193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:39.185118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eef76a-84fb-4109-834f-96f26794b018 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:39.192549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:39.193193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:39.193471 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:39.193930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:39.264122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:39.266435 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5df5c-aa7d-4277-814d-7465f96a5c01 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:39.278291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1104/2210] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:42.667724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:42.729516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f415ff8-baf8-4557-a616-341df9bb7e7a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:42.734687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:42.735310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:42.735583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:42.735851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] PATH_INFO: `/v3/users/695dfd31f7da435e8c6544b29a5298e1` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:42.737113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695dfd31f7da435e8c6544b29a5298e1)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:42.739003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:42.790757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695dfd31f7da435e8c6544b29a5298e1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:42.800334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695dfd31f7da435e8c6544b29a5298e1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:42.813200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695dfd31f7da435e8c6544b29a5298e1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:42.813950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695dfd31f7da435e8c6544b29a5298e1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:42.827222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695dfd31f7da435e8c6544b29a5298e1'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:42.833955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1107/2211] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:42 2022] DELETE /identity/v3/users/695dfd31f7da435e8c6544b29a5298e1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:42.838586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:42.839076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:42.839170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:42.859288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:42.860988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:42.906217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:42.926520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1105/2212] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:42.936278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:42.937797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:42.951363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:43.004419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63a59-3bef-40a2-bb9a-ba3212679526 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:43.008532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:43.009176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:43.009459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:43.009917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:43.024375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:43.026553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:43.038032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1108/2213] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:43.414043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:43.415615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:43.431017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122a6e8-8dd5-4057-88ae-54095eedd1a1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:43.435678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:43.436309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:43.436620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:43.436949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] PATH_INFO: `/v3/projects/b6d6514de9024d84bf73a0c0de580779` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:43.438083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d6514de9024d84bf73a0c0de580779)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:43.440138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:43.491524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d6514de9024d84bf73a0c0de580779', 'request_id': 'req-4ea8985a-1356-42d7-9a70-7bad6ccd494b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.500991 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d6514de9024d84bf73a0c0de580779', 'request_id': 'req-4ea8985a-1356-42d7-9a70-7bad6ccd494b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.507161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d6514de9024d84bf73a0c0de580779', 'request_id': 'req-4ea8985a-1356-42d7-9a70-7bad6ccd494b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.519228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] The token cache is being invalidate because project b6d6514de9024d84bf73a0c0de580779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:43.519508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.520164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.521730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1106/2214] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:43 2022] DELETE /identity/v3/projects/b6d6514de9024d84bf73a0c0de580779 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:43.525913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:43.535796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdc959-ec17-4411-8339-736bf019129c service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:43.597761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:43.598406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:43.598685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:43.598951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] PATH_INFO: `/v3/users/ced992120e514528a9288b6c1a42f59a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:43.600092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced992120e514528a9288b6c1a42f59a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:43.602087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:43.668365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced992120e514528a9288b6c1a42f59a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.677316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced992120e514528a9288b6c1a42f59a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.686914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced992120e514528a9288b6c1a42f59a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.687892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced992120e514528a9288b6c1a42f59a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.703250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ce5bdeb2-1704-4da9-a4d2-db50fa8306a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced992120e514528a9288b6c1a42f59a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:43.716519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1109/2215] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:43 2022] DELETE /identity/v3/users/ced992120e514528a9288b6c1a42f59a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:44.169900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.171879 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:44.185246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea8985a-1356-42d7-9a70-7bad6ccd494b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.261859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.262495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.262778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.263048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] PATH_INFO: `/v3/projects/99ff7a55ff27448f986518ff5499b4cc` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.264180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ff7a55ff27448f986518ff5499b4cc)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.266244 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.326751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ff7a55ff27448f986518ff5499b4cc', 'request_id': 'req-8109f003-b579-40b0-a93a-9358333f3985'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:44.336384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ff7a55ff27448f986518ff5499b4cc', 'request_id': 'req-8109f003-b579-40b0-a93a-9358333f3985'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:44.342407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ff7a55ff27448f986518ff5499b4cc', 'request_id': 'req-8109f003-b579-40b0-a93a-9358333f3985'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:44.350331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:44.356059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] The token cache is being invalidate because project 99ff7a55ff27448f986518ff5499b4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:44.356778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:44.357574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8109f003-b579-40b0-a93a-9358333f3985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:44.360313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1107/2216] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:44 2022] DELETE /identity/v3/projects/99ff7a55ff27448f986518ff5499b4cc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:44.366126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.366413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.366693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.387873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:44.389238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:44.423242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:44.463560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1110/2217] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:44.472819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.473357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.473508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.495407 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:44.496661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:44.507411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1108/2218] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:44.511950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.524091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f338c3-3279-4403-b30b-1a472687d499 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.530551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.531173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.531920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.532198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.532889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:44.533198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.534784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.548895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1111/2219] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:44.553313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.564743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29f7e2-bcf4-43f5-94af-95a1a05458a3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.569515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.570126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.570405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.570673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.571310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.573285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.603142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1109/2220] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:44 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:44.607423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.619895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639290f9-a391-48b8-8295-dbf17bb6ec9a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.696070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.697537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.697537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.697755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.698341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.700431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.728109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1112/2221] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:44 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:44.732863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.744873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c6599-9941-4674-b49e-2bcab3c9fd08 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.750236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.750889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.751145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.751531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.752083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.754151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.766055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1110/2222] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:44.771563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.782319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2bf9-f13f-4069-80ff-3f1bf6c4e93a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.787196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.787815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.788107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.788386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.789172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.790714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3edd-c124-423f-920e-61bae5b3e999 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.805686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1113/2223] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:44.811297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:44.823540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c859b-ae8d-4bf4-ab6e-c205a03e1a0e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:44.828975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:44.829682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.829963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.830273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] PATH_INFO: `/v3/projects/89718675597e421d8649fe932e6c9c18/users/4e52a072ad334693a697df8a4eef4089/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.848617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89718675597e421d8649fe932e6c9c18, user_id=4e52a072ad334693a697df8a4eef4089, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:44.853944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:44.873438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1111/2224] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:44 2022] PUT /identity/v3/projects/89718675597e421d8649fe932e6c9c18/users/4e52a072ad334693a697df8a4eef4089/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:44.899492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:44.901359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:44.902471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:44.945498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] MFA Rules not processed for user `4e52a072ad334693a697df8a4eef4089`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:44.947252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:45.030829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1114/2225] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:45.035499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.111760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bede7-7c67-453b-ba29-23928d43dbe1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.116981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.117597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.117895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.118173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.118833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.120918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.150747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1112/2226] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:45 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:36:45.156082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.168104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9c0f0-35b0-4667-a40f-487f7d1c9c2c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.202185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:45.243140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.243680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.243972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.244255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.244932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.246950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.271734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1115/2227] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:45 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:45.276795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.287890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0e23d-6db4-4b37-bdb3-9f03d2510679 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.293897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.294512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.295546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.295546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.297266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.298580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.311502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1113/2228] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:45.316420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.336979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37f650-a7bc-4f7f-a7ec-71cb36b5da3a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.345982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.346853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.347596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.347596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] PATH_INFO: `/v3/projects/a24dc0816f5f47b8816269279550d78e/users/44cea3298d8b491c95f624f1cd0e8a47/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.367190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24dc0816f5f47b8816269279550d78e, user_id=44cea3298d8b491c95f624f1cd0e8a47, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.370731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f71d76-d631-4908-9691-8dd1d3c2c412 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.401617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1116/2229] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:45 2022] PUT /identity/v3/projects/a24dc0816f5f47b8816269279550d78e/users/44cea3298d8b491c95f624f1cd0e8a47/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:45.407000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.423427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d972257-e9d1-45f6-98c0-53cffafa7cc0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.480980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.481270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.481595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.520851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.520851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.520851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.520851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.521216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.521216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.534635 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1114/2230] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:45.534838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:45.534838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:45.536239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.546526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:45.553729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35567386-de52-4db7-98f8-9e55b9d8ab6f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.559670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.560312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.560603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.560888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] PATH_INFO: `/v3/projects/a24dc0816f5f47b8816269279550d78e/users/44cea3298d8b491c95f624f1cd0e8a47/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.563949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24dc0816f5f47b8816269279550d78e, user_id=44cea3298d8b491c95f624f1cd0e8a47, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.567362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d247af7-3588-4573-acb0-53f2d279b5a2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.587804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1115/2231] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:45 2022] PUT /identity/v3/projects/a24dc0816f5f47b8816269279550d78e/users/44cea3298d8b491c95f624f1cd0e8a47/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:45.599486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.600337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.601154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.616986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1117/2232] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:36:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:45.638813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] MFA Rules not processed for user `44cea3298d8b491c95f624f1cd0e8a47`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:45.640722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:45.667280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.668870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:45.734018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1116/2233] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:45.745662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.747076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:45.756617 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca28be-714d-41ee-9d65-af512301a85a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.803229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:45.810764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c77070-a2de-4ece-a5c9-f5617c7f2fe7 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.813590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.814254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.814639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.815480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.825296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.826788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.828704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:45.829330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:45.829609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:45.829884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:45.838038 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1117/2234] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:45.906345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:45.908229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:45.921350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1118/2235] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:36:45.982297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:45.992069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2df537-df58-4961-bbce-a9757456010b service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:45.999511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:46.000538 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:46.004170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:46.004765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:46.089249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:46.090806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:46.107273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1118/2236] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:47.252228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:47.254839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:47.359803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039702c-bdef-4c9f-97d8-3155e9476548 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:47.365379 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:47.366130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:47.366574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:47.366936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:47.384877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:47.500216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:47.501246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad8608-3ec4-4a8b-b6b0-8584ca3327ea service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:47.544972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1119/2237] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:49.145873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:49.194608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:49.327615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77474-bf0b-4cc5-833a-f13b78c510cf service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:49.339537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:49.340955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:49.341854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:49.342959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] PATH_INFO: `/v3/users/4e52a072ad334693a697df8a4eef4089` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:49.345138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e52a072ad334693a697df8a4eef4089)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:49.348790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:49.442545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e52a072ad334693a697df8a4eef4089'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:49.477967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e52a072ad334693a697df8a4eef4089'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:49.487101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e52a072ad334693a697df8a4eef4089'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:49.488142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e52a072ad334693a697df8a4eef4089'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:49.488809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e52a072ad334693a697df8a4eef4089'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:49.510953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1119/2238] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:49 2022] DELETE /identity/v3/users/4e52a072ad334693a697df8a4eef4089 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:49.515670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:49.516665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:49.517174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:49.547134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:49.549481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:49.597141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:49.648862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1120/2239] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:49.656400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:49.657898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:49.731831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e4602-cd96-44d4-a45d-a36b0f49cf2a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:49.736319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:49.736979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:49.737265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:49.737522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:49.749731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:49.752193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:49.764748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1120/2240] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:50.158405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:50.160249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:50.191919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635706f-11e7-4595-94c1-6995c3be2a66 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:50.201620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:50.203310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:50.203816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:50.204338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] PATH_INFO: `/v3/projects/89718675597e421d8649fe932e6c9c18` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:50.205934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89718675597e421d8649fe932e6c9c18)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:50.208369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:50.284807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89718675597e421d8649fe932e6c9c18', 'request_id': 'req-02fa4e03-69e7-4228-9941-6372eba0a5f6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.298465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89718675597e421d8649fe932e6c9c18', 'request_id': 'req-02fa4e03-69e7-4228-9941-6372eba0a5f6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.328157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89718675597e421d8649fe932e6c9c18', 'request_id': 'req-02fa4e03-69e7-4228-9941-6372eba0a5f6'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.361896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] The token cache is being invalidate because project 89718675597e421d8649fe932e6c9c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:50.362842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.363815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.365749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1121/2241] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:50 2022] DELETE /identity/v3/projects/89718675597e421d8649fe932e6c9c18 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:50.369968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:50.388199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4bc79-3df4-44ac-8c5e-ecca3996fdb8 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:50.453547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:50.495336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:50.496114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:50.496646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:50.496646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] PATH_INFO: `/v3/users/44cea3298d8b491c95f624f1cd0e8a47` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:50.503906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44cea3298d8b491c95f624f1cd0e8a47)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:50.506541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:50.585018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44cea3298d8b491c95f624f1cd0e8a47'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.615851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44cea3298d8b491c95f624f1cd0e8a47'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.628262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44cea3298d8b491c95f624f1cd0e8a47'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.629774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44cea3298d8b491c95f624f1cd0e8a47'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.633065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-768dfe8e-c5e9-4bd5-961c-955680c835d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44cea3298d8b491c95f624f1cd0e8a47'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:50.667849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1121/2242] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:50 2022] DELETE /identity/v3/users/44cea3298d8b491c95f624f1cd0e8a47 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:51.431547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:51.435635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:51.466908 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa4e03-69e7-4228-9941-6372eba0a5f6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:51.571409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:51.572376 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:51.573030 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:51.573482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] PATH_INFO: `/v3/projects/a24dc0816f5f47b8816269279550d78e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:51.575165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24dc0816f5f47b8816269279550d78e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:51.578590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:51.678257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24dc0816f5f47b8816269279550d78e', 'request_id': 'req-aca38954-deb8-44b6-afe6-556b5d218c26'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:51.690281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24dc0816f5f47b8816269279550d78e', 'request_id': 'req-aca38954-deb8-44b6-afe6-556b5d218c26'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:51.692339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:51.728229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24dc0816f5f47b8816269279550d78e', 'request_id': 'req-aca38954-deb8-44b6-afe6-556b5d218c26'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:51.755048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] The token cache is being invalidate because project a24dc0816f5f47b8816269279550d78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:51.755703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:51.756755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-aca38954-deb8-44b6-afe6-556b5d218c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:51.765649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1122/2243] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:51 2022] DELETE /identity/v3/projects/a24dc0816f5f47b8816269279550d78e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:51.770869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:51.771989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:51.772282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:51.828438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:51.829802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:51.985560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1122/2244] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:51.987424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:51.987814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:51.988188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.025106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:52.033930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:52.052298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1123/2245] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:52.056245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.069481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad3f54-daf9-40c3-993a-26b8763c53c4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.088452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.089124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.089428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.090251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.090473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:52.090646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.092863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.141477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1123/2246] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:52.142312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.165217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b0485-6fea-4106-b223-efca8b1f2bce None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.179841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.180616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.180999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.181368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.182275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.184489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.233027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1124/2247] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:52 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:52.237792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.258483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583da93f-49cd-43a1-b7e6-aa1c07c0a2d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.387457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:52.414945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.415888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.416174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.416445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.417368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.419464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.461492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1124/2248] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:52 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:52.469734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.486461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07e1f-dd9d-44f8-847c-d1c1dfbbf83c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.492150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.493205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.493491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.493942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.494639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.496374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.512993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1125/2249] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:52.514307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.525751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a4921-3bc6-4910-a018-4294d917d5c1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.531794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.532425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.532742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.533360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.534218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.535972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6663a-805b-4c47-8f88-e97ee269715d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.558899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1125/2250] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:52.560120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.577082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ce444-bee9-4613-88a0-7d5048221ef1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.585272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.585961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.586528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.586814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] PATH_INFO: `/v3/projects/cd1d079ada2c4d74975af6b8b31e2a77/users/86f238451e8747f19ce80e3a3d8379ba/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.617968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1d079ada2c4d74975af6b8b31e2a77, user_id=86f238451e8747f19ce80e3a3d8379ba, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.622113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.658681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1126/2251] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:52 2022] PUT /identity/v3/projects/cd1d079ada2c4d74975af6b8b31e2a77/users/86f238451e8747f19ce80e3a3d8379ba/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:52.669821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e55059-89f1-41bd-8fb9-1a87d0612f1e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.670276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e55059-89f1-41bd-8fb9-1a87d0612f1e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.671197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e55059-89f1-41bd-8fb9-1a87d0612f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.709086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-43e55059-89f1-41bd-8fb9-1a87d0612f1e None None] MFA Rules not processed for user `86f238451e8747f19ce80e3a3d8379ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:52.710647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-43e55059-89f1-41bd-8fb9-1a87d0612f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:52.783254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1126/2252] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:36:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:52.798877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:52.801815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:52.896661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccc490-8da3-496d-adfb-58e0fbc53140 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:52.901896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:52.902723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:52.903009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:52.903317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:52.914600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:52.916050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be17d7-de70-4d6b-9a9b-fcec67c69e93 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:52.927285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1127/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:02.779496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:02.923631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=57 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:06.244789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.245366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.245477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.247011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1127/2254] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:37:06.254409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88660-92fa-4b5d-9a9a-7007cee5180f None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.254409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88660-92fa-4b5d-9a9a-7007cee5180f None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.254409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88660-92fa-4b5d-9a9a-7007cee5180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.277413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ccb88660-92fa-4b5d-9a9a-7007cee5180f None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:06.278890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ccb88660-92fa-4b5d-9a9a-7007cee5180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.378328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1128/2255] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:06.393301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:06.394378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.461577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3f7b7-7733-4429-b81b-d1717dc0d850 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:06.465443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:06.466051 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.466333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.466605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.477821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:06.479276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c50d78-2253-48be-a83d-fafaae0fa91a service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:06.490756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1128/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:06.572212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.572501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.572819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.574206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1129/2257] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:37:06.579867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4afc-f042-44e3-8ef3-114acb14413c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.580292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4afc-f042-44e3-8ef3-114acb14413c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.580524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b4afc-f042-44e3-8ef3-114acb14413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.601264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9b7b4afc-f042-44e3-8ef3-114acb14413c None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:06.602498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9b7b4afc-f042-44e3-8ef3-114acb14413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.662068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1129/2258] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:06.678716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:06.680648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.694551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260745f-3985-4020-8d10-e990f7ab660c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:06.698149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:06.698750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.699029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.699299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.706765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:06.708187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82218-f2fd-4672-b12c-964aee1eac7f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:06.716008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1130/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:06.802553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.802835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.803094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.804505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1130/2260] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:37:06.810665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.811600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.811885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.832516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:06.843332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.844742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1131/2261] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:37:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:06.856156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:06.858515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:06.872237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b435c-f10b-4095-aa7f-2e130d8fe6d8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:06.876509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:06.877620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:06.877620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:06.877853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:06.888068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:06.889569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a3123-7755-4d01-aa8f-cfbb82227095 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:06.901766 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1131/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:09.470110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:09.472330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:09.558269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969737c-4125-422a-ad8f-146ec6c1e17a None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:09.565947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:09.566570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:09.567022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:09.567395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:09.569794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:09.583396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:09.585285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:09.595834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1132/2263] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:12.370163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:12.370674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:12.370783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:12.391445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:12.392972 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:12.451556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1132/2264] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:37:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:12.465821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:12.467952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:12.543573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:12.549094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dedc0c9-383d-4762-8c1c-c01c09aa7f5a service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:12.554790 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:12.555408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:12.555712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:12.555962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:12.564564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:12.566172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549c96-5d3c-43a5-8bbd-1356d35af77d service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:12.574399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1133/2265] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:37:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:37:14.387935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:14.443854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:14.467007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daab55-29b6-4869-9589-f76da5eb80c6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:14.472654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:14.473344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:14.473782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:14.474060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] PATH_INFO: `/v3/users/86f238451e8747f19ce80e3a3d8379ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:14.475268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f238451e8747f19ce80e3a3d8379ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:14.477304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:14.538184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f238451e8747f19ce80e3a3d8379ba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:14.556176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f238451e8747f19ce80e3a3d8379ba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:14.566436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f238451e8747f19ce80e3a3d8379ba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:14.567462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f238451e8747f19ce80e3a3d8379ba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:14.568080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f238451e8747f19ce80e3a3d8379ba'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:14.584067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1133/2266] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:37:14 2022] DELETE /identity/v3/users/86f238451e8747f19ce80e3a3d8379ba => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:14.588493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:14.589101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:14.589101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:14.611322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:14.612760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:14.717103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1134/2267] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:14.728109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:14.729805 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:14.828901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636efa66-0451-4edd-87cb-a183d4a6dbb6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:14.833432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:14.834153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:14.834503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:14.834841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:14.847528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:14.849069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b80b-7d68-4567-a34e-afc9c5771071 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:14.861614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1134/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:15.311715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.313419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:15.341139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43271552-75e0-4b56-b448-85102ec7a4d7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.348115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.348817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.349097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.349370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] PATH_INFO: `/v3/projects/cd1d079ada2c4d74975af6b8b31e2a77` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.350520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1d079ada2c4d74975af6b8b31e2a77)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.352507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.418913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1d079ada2c4d74975af6b8b31e2a77', 'request_id': 'req-68a7caae-39af-4ee2-9348-127b1ee6c59e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:15.426757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1d079ada2c4d74975af6b8b31e2a77', 'request_id': 'req-68a7caae-39af-4ee2-9348-127b1ee6c59e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:15.437263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1d079ada2c4d74975af6b8b31e2a77', 'request_id': 'req-68a7caae-39af-4ee2-9348-127b1ee6c59e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:15.453343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] The token cache is being invalidate because project cd1d079ada2c4d74975af6b8b31e2a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:37:15.453631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:15.454493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68a7caae-39af-4ee2-9348-127b1ee6c59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:15.456638 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1135/2269] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:37:15 2022] DELETE /identity/v3/projects/cd1d079ada2c4d74975af6b8b31e2a77 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:15.462050 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.462930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.462930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.485786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:15.487032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:15.578094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1135/2270] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:15.587741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.588118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.588455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.616673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:15.617930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:15.627534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1136/2271] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:37:15.632322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.643082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593526-fcb7-42db-b8c8-ae8f34f4eb3d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.647994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.648633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.648920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.649195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.649852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:37:15.650155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.651769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.653496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:15.666996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1136/2272] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:37:15.671682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.682264 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07235b3f-6128-4a5c-9bee-2dcbb2cb90c0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.687656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.688369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.688915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.689446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.690287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.692369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.728352 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1137/2273] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:37:15 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:37:15.733170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.745291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dc47a-b1de-4af5-9e51-38503f9fd249 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.836197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.836802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.837094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.837384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.838046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.840206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.869220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1137/2274] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:37:15 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:37:15.874194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.885196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494d8d4-d06d-42fd-bf8c-f014c4c16deb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.889868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.890471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.890774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.891044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.891708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.893911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.905529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1138/2275] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:37:15.911587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.925754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e708d-00e3-4968-8a8d-01aee4034b8a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.931273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.931920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.932201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.932508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.933256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.935014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931c761-a6ac-4876-939f-57766679a596 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:15.948301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1138/2276] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:37:15.953195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:15.964303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda736-7d09-481b-8b48-b664d05e1dfb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:15.969102 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:15.969937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:15.970347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:15.970803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] PATH_INFO: `/v3/projects/8a6a9cadb5ce4d20920d97b96f81ff7a/users/0699141eded843e5a463034a9ea89942/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:15.988115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6a9cadb5ce4d20920d97b96f81ff7a, user_id=0699141eded843e5a463034a9ea89942, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:15.991820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:16.011934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1139/2277] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:37:15 2022] PUT /identity/v3/projects/8a6a9cadb5ce4d20920d97b96f81ff7a/users/0699141eded843e5a463034a9ea89942/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:16.026305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0c64-cbdb-4380-b7fd-a441f936f4e2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:16.026800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0c64-cbdb-4380-b7fd-a441f936f4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:16.027311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a0c64-cbdb-4380-b7fd-a441f936f4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:16.067668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a13a0c64-cbdb-4380-b7fd-a441f936f4e2 None None] MFA Rules not processed for user `0699141eded843e5a463034a9ea89942`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:16.068950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a13a0c64-cbdb-4380-b7fd-a441f936f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:16.174001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1139/2278] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:16.187025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:16.190254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:16.215032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:16.294214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe677-1400-46d7-9343-0b4bdf023238 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:16.298947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:16.299583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:16.299863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:16.300288 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:16.317210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:16.318840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:16.343010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1140/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:26.170691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:26.336550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=21 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:37.349574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853bea-42ee-4b80-a228-d7e8cb8a6a80 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:37.350089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853bea-42ee-4b80-a228-d7e8cb8a6a80 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:37.350198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853bea-42ee-4b80-a228-d7e8cb8a6a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:37.377413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-74853bea-42ee-4b80-a228-d7e8cb8a6a80 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:37.378654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-74853bea-42ee-4b80-a228-d7e8cb8a6a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:37.450723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1140/2280] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:37:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:37.465167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:37.467642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:37.567405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf9e40-6f93-42ef-b93d-54b51e742050 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:37.572915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:37.573576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:37.573854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:37.574121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:37.585564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:37.587206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:37.607667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1141/2281] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:37:47.442758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:47.593633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=32 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:51.446948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f8c8-92ab-432b-9d93-563f17c1f64c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:51.447539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f8c8-92ab-432b-9d93-563f17c1f64c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:51.447539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0f8c8-92ab-432b-9d93-563f17c1f64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:51.468722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-eab0f8c8-92ab-432b-9d93-563f17c1f64c None None] MFA Rules not processed for user `89b272427c6444b49293e5046c686ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:51.470084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-eab0f8c8-92ab-432b-9d93-563f17c1f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:51.547894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1141/2282] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:57.691431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:57.763280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30f703-7564-4cb5-9246-8273ee0f8f63 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:57.770235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:57.770965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:57.771235 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:57.771503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] PATH_INFO: `/v3/users/0699141eded843e5a463034a9ea89942` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:57.775684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0699141eded843e5a463034a9ea89942)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:57.778223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:57.836667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0699141eded843e5a463034a9ea89942'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:57.846180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0699141eded843e5a463034a9ea89942'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:57.854228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0699141eded843e5a463034a9ea89942'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:57.854960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0699141eded843e5a463034a9ea89942'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:57.883143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0699141eded843e5a463034a9ea89942'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:57.889541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1142/2283] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:37:57 2022] DELETE /identity/v3/users/0699141eded843e5a463034a9ea89942 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:57.894413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:57.894899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:57.894983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:57.918920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:57.920453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:57.989833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1142/2284] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:57.999396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:58.001169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:58.067710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bef1d-f809-4df1-a127-0c8758246ee7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:58.071453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:58.072089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.072325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.072616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.092485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:58.094173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7eb549-dc9a-41b8-b5db-18449ced7298 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:58.118720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1143/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:58.505755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:58.507552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:58.520672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d881-bbcc-4459-831a-8fb8f9c24600 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:58.525551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:58.526279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.526567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.527009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] PATH_INFO: `/v3/projects/8a6a9cadb5ce4d20920d97b96f81ff7a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.528856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6a9cadb5ce4d20920d97b96f81ff7a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:58.534513 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:58.584148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6a9cadb5ce4d20920d97b96f81ff7a', 'request_id': 'req-e18096f0-74a9-450d-acbe-1f739443b56d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:58.595941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6a9cadb5ce4d20920d97b96f81ff7a', 'request_id': 'req-e18096f0-74a9-450d-acbe-1f739443b56d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:58.602241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6a9cadb5ce4d20920d97b96f81ff7a', 'request_id': 'req-e18096f0-74a9-450d-acbe-1f739443b56d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:58.616839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] The token cache is being invalidate because project 8a6a9cadb5ce4d20920d97b96f81ff7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:37:58.617155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:58.618264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e18096f0-74a9-450d-acbe-1f739443b56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:58.620158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1143/2286] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:37:58 2022] DELETE /identity/v3/projects/8a6a9cadb5ce4d20920d97b96f81ff7a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:58.625587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.625894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.626126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.650065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:58.651273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:58.672504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:58.723127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1144/2287] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:58.732160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.732470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.732795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.755419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:58.757582 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:58.770952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1144/2288] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:58.777095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:58.796048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464592f8-fccb-40f1-8df6-e4dc8aea8f79 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:58.803445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:58.804389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.804929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.805403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.806583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:37:58.807114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:58.810069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:58.829553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1145/2289] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:37:58.834690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:58.845755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab4003-5510-413c-8540-f82908cacba6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:58.850810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:58.851428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.851708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.852222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.852918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:58.855705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:58.883749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:58.896369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1145/2290] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:37:58 2022] POST /identity/v3/projects => generated 397 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:37:58.901129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:58.912528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4553e-fd4f-4bde-b719-66fce0f69560 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:58.989784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:58.990319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:58.990600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:58.990874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:58.991515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:58.993483 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:59.019561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1146/2291] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:37:58 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:37:59.023956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:59.034842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac978c-af2c-4619-8920-5b0390d5b226 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:59.039783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:59.040388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:59.040703 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:59.040974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:59.041645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:59.043777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:59.055184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1146/2292] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:37:59.059941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:59.070524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401d113-2eaf-42bf-9b74-6c41830e1310 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:59.075273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:59.075923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:59.076202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:59.076477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:59.077162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:59.078762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a13aa9-375e-41a1-bb0e-a698b63c70cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:59.090114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1147/2293] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:37:59.094530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:59.105052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053686-8d90-4da8-8d59-f59b7f56195a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:59.110842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:59.111508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:59.111792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:59.112072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] PATH_INFO: `/v3/projects/eb84f6f443ea43d898e918e230a750df/users/4dd9dfcb19074d58b0dbf8895117026b/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:59.129057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb84f6f443ea43d898e918e230a750df, user_id=4dd9dfcb19074d58b0dbf8895117026b, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:59.131542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:59.145550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1147/2294] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:37:59 2022] PUT /identity/v3/projects/eb84f6f443ea43d898e918e230a750df/users/4dd9dfcb19074d58b0dbf8895117026b/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:59.217310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:59.217672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:59.218134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:59.248951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] MFA Rules not processed for user `4dd9dfcb19074d58b0dbf8895117026b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:59.250565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:59.333550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1148/2295] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:37:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:59.346918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:59.349249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:59.426536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bf0af-6c11-4a7b-b8ae-b34fc50d0183 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:59.430353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:59.431072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:59.431446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:59.431777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:59.442618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:59.444088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c9e86-becc-4fb7-8cbf-edef596336ba service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:59.456715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1148/2296] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:38:06.787359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:06.806133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:06.851961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0beb9-7b35-4cdf-8910-b87887b06f86 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:06.856610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:06.857248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:06.857524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:06.857794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] PATH_INFO: `/v3/users/4dd9dfcb19074d58b0dbf8895117026b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:06.858920 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd9dfcb19074d58b0dbf8895117026b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:06.860910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:06.917677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd9dfcb19074d58b0dbf8895117026b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:06.925717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd9dfcb19074d58b0dbf8895117026b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:06.932689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd9dfcb19074d58b0dbf8895117026b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:06.933931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd9dfcb19074d58b0dbf8895117026b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:06.945753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd9dfcb19074d58b0dbf8895117026b'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:06.952249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1149/2297] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:38:06 2022] DELETE /identity/v3/users/4dd9dfcb19074d58b0dbf8895117026b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:38:06.956648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:06.957121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:06.957258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:06.976029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:38:06.978705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:07.038379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1149/2298] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:38:07.049018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:07.050537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:07.115272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa21384-99f2-415c-89f8-c7b23df3f91c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:07.119178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:07.119802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:07.120084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:07.120360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:07.131892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:07.133378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5c52f-873d-46a9-9686-734c1180f1b4 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:07.143798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1150/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:38:07.428421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:07.429915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:07.448275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c20f3-5fa2-4905-9f8a-4df6746a5992 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:07.455649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:07.456215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:07.456501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:07.456802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] PATH_INFO: `/v3/projects/eb84f6f443ea43d898e918e230a750df` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:07.457883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb84f6f443ea43d898e918e230a750df)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:07.459790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:07.512442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb84f6f443ea43d898e918e230a750df', 'request_id': 'req-1883a689-18ab-4173-881e-304a01fba160'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:07.521479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb84f6f443ea43d898e918e230a750df', 'request_id': 'req-1883a689-18ab-4173-881e-304a01fba160'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:07.526615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:07.528469 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb84f6f443ea43d898e918e230a750df', 'request_id': 'req-1883a689-18ab-4173-881e-304a01fba160'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:07.543920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] The token cache is being invalidate because project eb84f6f443ea43d898e918e230a750df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:38:07.544302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:07.545000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1883a689-18ab-4173-881e-304a01fba160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:38:07.546511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1150/2300] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:38:07 2022] DELETE /identity/v3/projects/eb84f6f443ea43d898e918e230a750df => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:38:08.781354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:08.781932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:08.782151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:08.802966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] MFA Rules not processed for user `5344cf7b391d44beac2c602ac3e2e64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:38:08.804041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:08.842726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:08.873460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1151/2301] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:38:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:38:17.539902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:18.864338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:36.453253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65a94-c3e8-4bd6-ad15-773a0ad41b8c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:36.453762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65a94-c3e8-4bd6-ad15-773a0ad41b8c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:36.453855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65a94-c3e8-4bd6-ad15-773a0ad41b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:36.473130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d2f65a94-c3e8-4bd6-ad15-773a0ad41b8c None None] MFA Rules not processed for user `55a5c6359109440fbb66d985fafc24bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:38:36.474325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d2f65a94-c3e8-4bd6-ad15-773a0ad41b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:36.533580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1151/2302] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:38:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:38:46.526400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=25 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:01.989993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:01.991546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:02.093848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422260b-a166-42ce-ab4b-efa6a90941c4 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:02.098531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:02.099289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:02.099573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:02.099849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:02.101802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:02.201755 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:02.203640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:02.218406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1152/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:03.835503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c895c98-b4c0-4a33-80dd-e531493e09e6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:03.836008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c895c98-b4c0-4a33-80dd-e531493e09e6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:03.836115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c895c98-b4c0-4a33-80dd-e531493e09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:03.855716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0c895c98-b4c0-4a33-80dd-e531493e09e6 None None] MFA Rules not processed for user `6f0d97b196f44ec985db30f86d4ad3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:03.857091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0c895c98-b4c0-4a33-80dd-e531493e09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:03.913452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1152/2304] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:04.686804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:04.699154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d858c96-6cac-4d38-9f3c-8333c4dcb59e service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:04.707172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:04.707906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.708857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.709657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.713125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:04.735792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:04.737348 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:04.750478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1153/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:11.349834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:11.350353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:11.350459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:11.370437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] MFA Rules not processed for user `89b272427c6444b49293e5046c686ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:11.372743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:11.428519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1153/2306] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:14.745326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=60 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:18.021540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:18.032944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fe328-243b-4a3b-a089-6fac52a628ea service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:18.063100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:18.066213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:18.066828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:18.070203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:18.077414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:18.096865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:18.098857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613da713-984d-4f9c-87e4-06eb487d9cf3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:18.116990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1154/2307] 10.222.0.20 () {60 vars in 1396 bytes} [Mon Aug 29 21:39:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:19.025226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:19.026744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:19.048298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d45dad-b435-49ef-8ce4-1f759489e724 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:19.054729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:19.055618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:19.056124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:19.057312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:19.059554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:19.071777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:19.074763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:19.086173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1154/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:19.428643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:19.429343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:19.429343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:19.455891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] MFA Rules not processed for user `d882c87d5de7497b822ec7b1798d2008`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:19.457320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:19.519821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1155/2309] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:25.762098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:25.771872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25ce33-0b63-4203-b748-26792d61999e service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:25.775512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:25.776116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:25.776397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:25.776704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:25.778669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:25.845517 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:25.846968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:25.860439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1155/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:26.590597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:26.607854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6181cac-0256-4a6c-9108-370542b1c001 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:26.615595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:26.616512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:26.616827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:26.617096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:26.618865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:26.630669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:26.632250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:26.645526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1156/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:30.022642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:30.025116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:30.033171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:30.039420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500dc6c-1131-4ea7-afe9-32e7c5a1e53d service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:30.045378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:30.046156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.046438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.046825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] PATH_INFO: `/v3/users/5344cf7b391d44beac2c602ac3e2e64b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.047954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5344cf7b391d44beac2c602ac3e2e64b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:30.049965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:30.098108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5344cf7b391d44beac2c602ac3e2e64b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.111395 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5344cf7b391d44beac2c602ac3e2e64b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.118365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5344cf7b391d44beac2c602ac3e2e64b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.119121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5344cf7b391d44beac2c602ac3e2e64b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.119739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1d96900c-1cf2-4357-8e6f-b00c948b5281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5344cf7b391d44beac2c602ac3e2e64b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.133725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1156/2312] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:30 2022] DELETE /identity/v3/users/5344cf7b391d44beac2c602ac3e2e64b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:30.491750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:30.504538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dc2e2-7221-42fb-81ce-38841a66924b service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:30.566531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:30.567088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.567366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.567637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] PATH_INFO: `/v3/projects/57c7abc43eb142d3b36a6d91beee5d10` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.568812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c7abc43eb142d3b36a6d91beee5d10)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:30.570697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:30.617664 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c7abc43eb142d3b36a6d91beee5d10', 'request_id': 'req-86eab53c-18d7-476d-83d2-7ddd4caba56e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.623282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c7abc43eb142d3b36a6d91beee5d10', 'request_id': 'req-86eab53c-18d7-476d-83d2-7ddd4caba56e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.630500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c7abc43eb142d3b36a6d91beee5d10', 'request_id': 'req-86eab53c-18d7-476d-83d2-7ddd4caba56e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.642630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] The token cache is being invalidate because project 57c7abc43eb142d3b36a6d91beee5d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:30.642922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.643565 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-86eab53c-18d7-476d-83d2-7ddd4caba56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:30.645106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1157/2313] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:30 2022] DELETE /identity/v3/projects/57c7abc43eb142d3b36a6d91beee5d10 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:30.650989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.651284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.651561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.674268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:30.675597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:30.738819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1157/2314] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:30.749404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.749404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.749404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.770008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:30.771250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:30.780375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1158/2315] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:30.784699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:30.798665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b456ec2-565e-40fd-9c90-1fad216ade85 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:30.807002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:30.808066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.808784 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.809280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.810258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:30.810752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:30.812531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:30.828815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1158/2316] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:30.830197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:30.834786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:30.841669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6cdb2-e4b7-4318-a9ab-cc2f66a4f9e9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:30.851589 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:30.852215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.852536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.852864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.853508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:30.855501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:30.886569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1159/2317] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:30 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:30.891476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:30.902120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579368c7-c298-49c0-9737-2c5509611324 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:30.963989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:30.964964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:30.964964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:30.965285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:30.965834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:30.967746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:30.977287 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:30.997720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1159/2318] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:30 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:39:30.999043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.012277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc88c8-af29-4480-80dc-8a03f3007e3d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.017758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.018366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.018644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.018913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.019560 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.021233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.032974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1160/2319] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:31.036622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.047775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2dc6c-130c-4677-96dd-7c34c4a0534c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.052612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.053219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.053498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.053770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.054419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.056661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c945f7-c7b8-45a8-a43d-139d2e00ff35 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.068058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1160/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:31.072521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.083428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f046f-22b3-4e79-af72-bb77d8fbb522 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.088602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.089286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.089568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.089846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] PATH_INFO: `/v3/projects/0b7b6b9a32e74bf49588c8056c1722bf/users/498b1fbe4871479eaa82841843f02e81/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.106903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7b6b9a32e74bf49588c8056c1722bf, user_id=498b1fbe4871479eaa82841843f02e81, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.109600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.123223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1161/2321] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:31 2022] PUT /identity/v3/projects/0b7b6b9a32e74bf49588c8056c1722bf/users/498b1fbe4871479eaa82841843f02e81/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:39:31.140173 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.140453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.140757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.169268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] MFA Rules not processed for user `498b1fbe4871479eaa82841843f02e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:31.170458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:31.236216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1161/2322] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:31.240718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.307907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a941e-45c3-421d-9fd0-d77484f8cc12 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.313882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.314504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.314812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.315110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.315796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.317802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.346746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1162/2323] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:31 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:31.350861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.362926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c4c3f-253b-46bb-8b78-893336b4ebb5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.426015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.426641 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.426922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.427217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.427856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.429960 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.456688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1162/2324] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:31 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:31.459716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.473232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2c265-931b-4441-aee7-3b104c2e33cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.479140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.479752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.480029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.480686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.481036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.482868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.496506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1163/2325] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:31.501195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.513687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075382a2-a3b7-4e7b-a258-086150a9e3cb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.522075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.522918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.523314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.523654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] PATH_INFO: `/v3/projects/966af1dd107347dca02d9ba61de740e8/users/2e0a0347f457432f854fcb7e63ec9573/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.545580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966af1dd107347dca02d9ba61de740e8, user_id=2e0a0347f457432f854fcb7e63ec9573, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.549159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.569955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1163/2326] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:31 2022] PUT /identity/v3/projects/966af1dd107347dca02d9ba61de740e8/users/2e0a0347f457432f854fcb7e63ec9573/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:31.574855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.586202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184fe8a-598f-46c8-9088-091008ac72a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.676932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.677903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.678363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.678806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.679919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.682896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183c113-c806-4ff6-a46f-bf3dc39de991 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.699042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1164/2327] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:31.701246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.721074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e53f0-40fa-416a-b6e1-5ef9185da615 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.730274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.731501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.732008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.732515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] PATH_INFO: `/v3/projects/966af1dd107347dca02d9ba61de740e8/users/2e0a0347f457432f854fcb7e63ec9573/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.747358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966af1dd107347dca02d9ba61de740e8, user_id=2e0a0347f457432f854fcb7e63ec9573, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:31.752995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:31.768099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1164/2328] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:31 2022] PUT /identity/v3/projects/966af1dd107347dca02d9ba61de740e8/users/2e0a0347f457432f854fcb7e63ec9573/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:31.781332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.781624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.781904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.811626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] MFA Rules not processed for user `2e0a0347f457432f854fcb7e63ec9573`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:31.812865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:31.817063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:31.900766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1165/2329] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:31 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:31.917547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.919990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:31.947265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:31.982569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707af49-7161-4a35-ab7c-94d2b68707f5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.986600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.987223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.987516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.987934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.999036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:32.000607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e510a-a9fd-495e-a7d8-f52c94226ee0 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:32.012135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1165/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:32.185045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:32.267050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9f93f-9d54-4dcc-a540-d684f09fdeb0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:32.271946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:32.272614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:32.272901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:32.273178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] PATH_INFO: `/v3/users/498b1fbe4871479eaa82841843f02e81` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:32.274869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498b1fbe4871479eaa82841843f02e81)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:32.276846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:32.340712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498b1fbe4871479eaa82841843f02e81'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:32.348856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498b1fbe4871479eaa82841843f02e81'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:32.359152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498b1fbe4871479eaa82841843f02e81'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:32.359940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498b1fbe4871479eaa82841843f02e81'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:32.374748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498b1fbe4871479eaa82841843f02e81'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:32.380836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1166/2331] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:32 2022] DELETE /identity/v3/users/498b1fbe4871479eaa82841843f02e81 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:32.385388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:32.385873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:32.385974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:32.405886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:32.407095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:32.468209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1166/2332] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:32.478386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:32.479853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:32.503294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:32.544384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b65f-24bc-433c-8ea9-e609e13dab20 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:32.547997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:32.548634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:32.548916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:32.549185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:32.559675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:32.561776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:32.572915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1167/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:32.915790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:32.917288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:32.935233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de027d-775b-48bc-9756-f94520e2aba3 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:32.940939 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:32.942020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:32.942335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:32.942611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] PATH_INFO: `/v3/projects/0b7b6b9a32e74bf49588c8056c1722bf` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:32.943890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7b6b9a32e74bf49588c8056c1722bf)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:32.945742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:32.994250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7b6b9a32e74bf49588c8056c1722bf', 'request_id': 'req-2f02add7-1df0-48e5-bb66-eefeacdd7710'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.002066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7b6b9a32e74bf49588c8056c1722bf', 'request_id': 'req-2f02add7-1df0-48e5-bb66-eefeacdd7710'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.008409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7b6b9a32e74bf49588c8056c1722bf', 'request_id': 'req-2f02add7-1df0-48e5-bb66-eefeacdd7710'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.020053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] The token cache is being invalidate because project 0b7b6b9a32e74bf49588c8056c1722bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:33.020339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.021015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.022508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1167/2334] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:32 2022] DELETE /identity/v3/projects/0b7b6b9a32e74bf49588c8056c1722bf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:33.027081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:33.036739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ab117-d10b-493c-948c-16b495c4b19a service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:33.106059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:33.107222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.107222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.107458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] PATH_INFO: `/v3/users/2e0a0347f457432f854fcb7e63ec9573` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.108615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0a0347f457432f854fcb7e63ec9573)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:33.110573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:33.157868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:33.160755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:33.173449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0a0347f457432f854fcb7e63ec9573'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.183480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0a0347f457432f854fcb7e63ec9573'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.192238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0a0347f457432f854fcb7e63ec9573'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.193048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0a0347f457432f854fcb7e63ec9573'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.210752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0a0347f457432f854fcb7e63ec9573'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.218541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1168/2335] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:33 2022] DELETE /identity/v3/users/2e0a0347f457432f854fcb7e63ec9573 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:33.237493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:33.242878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02add7-1df0-48e5-bb66-eefeacdd7710 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:33.330332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:33.331247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.331586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.331970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.334222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:33.347461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:33.349182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81addd9-4c37-4949-8689-1609e995bbd6 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:33.359993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1168/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:33.588383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:33.591255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:33.655300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:33.686297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103e96c-faea-4b95-9173-8cd6c8af58ed admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:33.691922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:33.692213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.692493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.692797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] PATH_INFO: `/v3/projects/966af1dd107347dca02d9ba61de740e8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.693937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=966af1dd107347dca02d9ba61de740e8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:33.695877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:33.746578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966af1dd107347dca02d9ba61de740e8', 'request_id': 'req-67de9e39-505c-4d2b-bd62-25cb155f7f7d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.752885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966af1dd107347dca02d9ba61de740e8', 'request_id': 'req-67de9e39-505c-4d2b-bd62-25cb155f7f7d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.762019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966af1dd107347dca02d9ba61de740e8', 'request_id': 'req-67de9e39-505c-4d2b-bd62-25cb155f7f7d'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.775945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] The token cache is being invalidate because project 966af1dd107347dca02d9ba61de740e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:33.776296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.777152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67de9e39-505c-4d2b-bd62-25cb155f7f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:33.779237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1169/2337] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:33 2022] DELETE /identity/v3/projects/966af1dd107347dca02d9ba61de740e8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:33.784103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.784692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.784692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.806023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:33.807242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:33.888031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1169/2338] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:33.890072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.890355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.890625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.913477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:33.914780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:33.925205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1170/2339] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:33.929443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:33.939366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c6c51-8555-4efd-a387-668a27f3f350 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:33.944208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:33.944840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.945118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.945391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.946092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:33.946394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:33.947977 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:33.960356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1170/2340] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:33.965258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:33.975461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2d1c5-c586-4247-82e3-f32389056198 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:33.982214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:33.983044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:33.983189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:33.983464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:33.984112 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:33.986145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.020945 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1171/2341] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:33 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:34.025460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.037494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306cd1f-efd0-4c1a-8560-bd2e8df52198 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.126683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.127327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.127604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.128030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.128708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.130629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.159274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1171/2342] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:34 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:39:34.160924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.176744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f5fe-bf12-4263-b73c-511598ff5902 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.183594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.184369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.184789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.185170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.186403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.191754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.204673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1172/2343] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:34.208610 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.220436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c512a-d1d0-4249-ade9-d213fbd8bc5b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.226604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.227271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.227563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.228157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.229030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.230704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce564b-a7e4-465e-9879-664c277bfd8f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.244694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1172/2344] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:34.248937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.261832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d2e3-ead1-4cf3-95f8-b89a0b415733 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.268512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.269676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.270130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.270404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] PATH_INFO: `/v3/projects/cb943d78e0894e219ba68b017b475cf2/users/02af28315dd74384931cfca369b767f4/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.302784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb943d78e0894e219ba68b017b475cf2, user_id=02af28315dd74384931cfca369b767f4, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.304140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.325649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1173/2345] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:34 2022] PUT /identity/v3/projects/cb943d78e0894e219ba68b017b475cf2/users/02af28315dd74384931cfca369b767f4/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:39:34.336448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.337893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.338547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.373774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:34.377669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] MFA Rules not processed for user `02af28315dd74384931cfca369b767f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:34.381189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:34.468871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1173/2346] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:34.473355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.561912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756110a-bf18-4935-956f-ea8a1af41799 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.567147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.567835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.568700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.568700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.569306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.571355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.587780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:34.612065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1174/2347] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:34 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:34.615765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.626910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906408c2-5a1e-41e4-970e-b054e6d2591c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.685765 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.686371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.686645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.686914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.687550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.689485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.715973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1174/2348] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:34 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:34.717758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.729449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45113173-f8b4-41ae-8240-03eb458f49d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.734566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.735173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.735457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.735731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.736372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.738025 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.748761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1175/2349] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:34.753094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.762968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaee6a-a25f-48e7-a8e9-62bd95d5af12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.767891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.768474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.768808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.769081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] PATH_INFO: `/v3/projects/78bb7e2aa2954760a7ceedc2e0349760/users/b64543895e454f44bc0ae2f260e1ef38/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.785941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bb7e2aa2954760a7ceedc2e0349760, user_id=b64543895e454f44bc0ae2f260e1ef38, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.788616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.802691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1175/2350] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:34 2022] PUT /identity/v3/projects/78bb7e2aa2954760a7ceedc2e0349760/users/b64543895e454f44bc0ae2f260e1ef38/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:34.806855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.816783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc969890-c2da-42d5-bba3-138f95a5c3ea admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.874880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.875491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.875773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.876046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.876719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.878325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f0572-f1ba-47ad-8265-536f08e0082f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.888886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1176/2351] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:34.893675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:34.905030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f890f-4e09-4a01-b5c0-de3f965c643c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:34.909722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:34.910447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.910725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.910997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] PATH_INFO: `/v3/projects/78bb7e2aa2954760a7ceedc2e0349760/users/b64543895e454f44bc0ae2f260e1ef38/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.912907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78bb7e2aa2954760a7ceedc2e0349760, user_id=b64543895e454f44bc0ae2f260e1ef38, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:34.915524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:34.926876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1176/2352] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:34 2022] PUT /identity/v3/projects/78bb7e2aa2954760a7ceedc2e0349760/users/b64543895e454f44bc0ae2f260e1ef38/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:34.938195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:34.938480 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:34.938753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:34.968217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] MFA Rules not processed for user `b64543895e454f44bc0ae2f260e1ef38`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:34.969467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:35.046691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1177/2353] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:35.060770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:35.062695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:35.123341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c3af-6c94-446e-8b6f-29add11a9eb6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:35.126969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:35.127571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:35.127959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:35.128231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:35.137429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:35.138885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:35.143144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:35.151516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1177/2354] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:35.479146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:35.488769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e886b7a-0cc3-4f1f-9cf1-25c127d1f0e5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:35.493444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b64543895e454f44bc0ae2f260e1ef38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78bb7e2aa2954760a7ceedc2e0349760', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:35.494048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:35.494330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:35.494606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:35.495247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:35.495550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:35.497169 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:35.508846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1178/2355] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:35.513418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:35.522664 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a702e1-8a81-4def-9165-0c526eecb93c service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:35.527821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b64543895e454f44bc0ae2f260e1ef38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78bb7e2aa2954760a7ceedc2e0349760', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:35.528414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:35.528724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:35.528998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:35.529650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:35.531531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:35.558809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1178/2356] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:35 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:35.563186 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:35.574031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038e2a6-f0e4-493a-904d-72ae4b40675f tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:35.623238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:35.636689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b64543895e454f44bc0ae2f260e1ef38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78bb7e2aa2954760a7ceedc2e0349760', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:35.637598 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:35.637882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:35.638160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:35.638826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:35.639393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:35.642286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:35.656747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1179/2357] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:35.692946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:35.703861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96514c-3b91-4102-8368-162c1c5f758c tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:35.708609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b64543895e454f44bc0ae2f260e1ef38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78bb7e2aa2954760a7ceedc2e0349760', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:35.709310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:35.709595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:35.709870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] PATH_INFO: `/v3/projects/17da244778524eb1ac8ec47e2681eab0` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:35.710982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorizing `identity:delete_project(project_id=17da244778524eb1ac8ec47e2681eab0)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:35.713534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:35.771086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17da244778524eb1ac8ec47e2681eab0', 'request_id': 'req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:35.786436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17da244778524eb1ac8ec47e2681eab0', 'request_id': 'req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:35.794055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17da244778524eb1ac8ec47e2681eab0', 'request_id': 'req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:35.807562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] The token cache is being invalidate because project 17da244778524eb1ac8ec47e2681eab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:35.807852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:35.808887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-564ecbcc-6c91-46a5-a0fa-3cf5c7b72a2d tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:35.810971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1179/2358] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:35 2022] DELETE /identity/v3/projects/17da244778524eb1ac8ec47e2681eab0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:36.006710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:36.008507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:36.074610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57b451-cf78-4812-9825-4fc27853c0dc tempest-VolumeQuotaClassesTest-1540863853 tempest-VolumeQuotaClassesTest-1540863853-project-admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:36.131546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:36.132176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:36.132452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:36.132766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] PATH_INFO: `/v3/users/02af28315dd74384931cfca369b767f4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:36.134452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02af28315dd74384931cfca369b767f4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:36.136721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:36.184760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02af28315dd74384931cfca369b767f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.192263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02af28315dd74384931cfca369b767f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.200263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02af28315dd74384931cfca369b767f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.201149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02af28315dd74384931cfca369b767f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.216350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02af28315dd74384931cfca369b767f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.223568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1180/2359] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:36 2022] DELETE /identity/v3/users/02af28315dd74384931cfca369b767f4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:36.228092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:36.228381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:36.228681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:36.248302 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:36.249549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:36.314573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1180/2360] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:36.324260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:36.325798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:36.338962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:36.388409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d650bad-5bfc-4d58-a718-ff7bf4354c5b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:36.393691 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:36.394324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:36.394592 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:36.394947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:36.405004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:36.406460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:36.417831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1181/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:36.804848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:36.806856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:36.820912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e239-6acc-43fb-90f7-53d00114d705 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:36.827065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:36.827815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:36.828103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:36.828555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] PATH_INFO: `/v3/projects/cb943d78e0894e219ba68b017b475cf2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:36.830220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb943d78e0894e219ba68b017b475cf2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:36.832274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:36.906422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb943d78e0894e219ba68b017b475cf2', 'request_id': 'req-3f72058c-2857-472b-8d35-ce9974a0f234'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.910507 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:36.915485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb943d78e0894e219ba68b017b475cf2', 'request_id': 'req-3f72058c-2857-472b-8d35-ce9974a0f234'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.923115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb943d78e0894e219ba68b017b475cf2', 'request_id': 'req-3f72058c-2857-472b-8d35-ce9974a0f234'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.935574 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] The token cache is being invalidate because project cb943d78e0894e219ba68b017b475cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:36.935867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.936544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:36.938098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1181/2362] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:36 2022] DELETE /identity/v3/projects/cb943d78e0894e219ba68b017b475cf2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:36.942094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:36.952441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdb9ea-4672-4fd3-b0da-c61a11607dad service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:37.013190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:37.013827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.014381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.014653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] PATH_INFO: `/v3/users/b64543895e454f44bc0ae2f260e1ef38` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.015789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64543895e454f44bc0ae2f260e1ef38)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:37.017744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:37.067558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64543895e454f44bc0ae2f260e1ef38'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.078492 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64543895e454f44bc0ae2f260e1ef38'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.088168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64543895e454f44bc0ae2f260e1ef38'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.088912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64543895e454f44bc0ae2f260e1ef38'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.106107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-61916cb4-88e4-4b9e-86ae-915115bbf001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64543895e454f44bc0ae2f260e1ef38'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.112499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1182/2363] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:36 2022] DELETE /identity/v3/users/b64543895e454f44bc0ae2f260e1ef38 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:37.530657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:37.532200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:37.545431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72058c-2857-472b-8d35-ce9974a0f234 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:37.616571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:37.617186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.617470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.617750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] PATH_INFO: `/v3/projects/78bb7e2aa2954760a7ceedc2e0349760` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.619378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78bb7e2aa2954760a7ceedc2e0349760)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:37.622317 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:37.671606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78bb7e2aa2954760a7ceedc2e0349760', 'request_id': 'req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.679602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78bb7e2aa2954760a7ceedc2e0349760', 'request_id': 'req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.686499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78bb7e2aa2954760a7ceedc2e0349760', 'request_id': 'req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.698171 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] The token cache is being invalidate because project 78bb7e2aa2954760a7ceedc2e0349760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:37.698573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.699427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-82057b11-b71a-4736-8f3c-bd6b4c9e2da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:37.700936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1182/2364] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:37 2022] DELETE /identity/v3/projects/78bb7e2aa2954760a7ceedc2e0349760 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:37.705868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.706226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.706628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.729284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:37.730623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:37.762704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:37.798418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1183/2365] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:37.806677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596903cf-0f73-4421-b99f-42748801b132 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.807220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596903cf-0f73-4421-b99f-42748801b132 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.807220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596903cf-0f73-4421-b99f-42748801b132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.829500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-596903cf-0f73-4421-b99f-42748801b132 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:37.830818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-596903cf-0f73-4421-b99f-42748801b132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:37.843198 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1183/2366] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:37.844458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:37.854479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6381d-9cd2-4b7a-b747-34c912f676b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:37.864833 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:37.865499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.865821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.866097 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.866844 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:37.867087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:37.868780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:37.885586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1184/2367] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:37.887255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596903cf-0f73-4421-b99f-42748801b132 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:37.897545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596903cf-0f73-4421-b99f-42748801b132 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:37.903829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:37.904559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:37.904995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:37.905356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:37.906079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:37.908067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:37.939024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1184/2368] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:37 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:39:37.943473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:37.955836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728cac5-9a41-436d-9a75-8a7d45af6104 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.015786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.016397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.016864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.017147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.017791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.019703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.044363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1185/2369] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:37 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:39:38.048931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.061902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf326c-d04f-4549-a0a2-15e7a88cecaa admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.066597 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.067204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.067479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.067746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.068388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.070031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.080667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1185/2370] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:38.084842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.095383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b397-b70d-4515-a19a-9e776f7c13c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.100128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.100780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.101067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.101343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.102760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.104077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4cb14-7490-41c6-b6bb-dc8b2161dcbb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.121782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1186/2371] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:38.122957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.134401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1864f12-9fd1-4c3d-a7e9-96307cf65949 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.140252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.141182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.141458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.141742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] PATH_INFO: `/v3/projects/3f19f70b1f5545aca8038532401fe71e/users/ab99873de96c4961a769ea7feb32d224/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.158590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19f70b1f5545aca8038532401fe71e, user_id=ab99873de96c4961a769ea7feb32d224, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.161391 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.179138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1186/2372] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:38 2022] PUT /identity/v3/projects/3f19f70b1f5545aca8038532401fe71e/users/ab99873de96c4961a769ea7feb32d224/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:38.194471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.194754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.195035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.223810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] MFA Rules not processed for user `ab99873de96c4961a769ea7feb32d224`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:38.225050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:38.296849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1187/2373] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:38.308220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.310311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:38.323348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:38.383775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e5832-b39f-45b6-9125-ea3f2c12d535 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.388067 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.388733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.389297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.389567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.398052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.399981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4878a5-6c81-43cd-a63c-9378dd56628c service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.409545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1187/2374] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:38.431925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.464815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:38.500865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6ab1c-e37c-4e71-a9cd-21724747e4bd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.505428 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.506067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.506450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.506745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] PATH_INFO: `/v3/users/ab99873de96c4961a769ea7feb32d224` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.507893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab99873de96c4961a769ea7feb32d224)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.509929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.564734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab99873de96c4961a769ea7feb32d224'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:38.573662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab99873de96c4961a769ea7feb32d224'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:38.581006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab99873de96c4961a769ea7feb32d224'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:38.581671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab99873de96c4961a769ea7feb32d224'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:38.595624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab99873de96c4961a769ea7feb32d224'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:38.602049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1188/2375] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:38 2022] DELETE /identity/v3/users/ab99873de96c4961a769ea7feb32d224 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:38.606933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.607221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.607499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.627314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:38.628526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:38.690941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1188/2376] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:38.701228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:38.702956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:38.766731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73661c-d707-4d0b-8915-157952e8d855 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:38.770408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:38.771017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:38.771291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:38.771561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:38.781901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:38.783325 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd45aad-e3b8-46ec-9fc9-5061edac6b9f service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:38.793459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1189/2377] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:39.136183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.137809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:39.155299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861fa90-922c-4981-b995-8021b92c77ca None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.161151 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.162022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.162299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.162570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] PATH_INFO: `/v3/projects/3f19f70b1f5545aca8038532401fe71e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.163756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f19f70b1f5545aca8038532401fe71e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.166862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.226598 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f19f70b1f5545aca8038532401fe71e', 'request_id': 'req-81f8ef7e-c541-4876-b3f7-1240def55fdd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:39.236337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f19f70b1f5545aca8038532401fe71e', 'request_id': 'req-81f8ef7e-c541-4876-b3f7-1240def55fdd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:39.246100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f19f70b1f5545aca8038532401fe71e', 'request_id': 'req-81f8ef7e-c541-4876-b3f7-1240def55fdd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:39.259358 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] The token cache is being invalidate because project 3f19f70b1f5545aca8038532401fe71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:39.260120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:39.260314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-81f8ef7e-c541-4876-b3f7-1240def55fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:39.262872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1189/2378] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:39 2022] DELETE /identity/v3/projects/3f19f70b1f5545aca8038532401fe71e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:39.268425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.269770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.269770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.294087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:39.295477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:39.347249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:39.371723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1190/2379] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:39.373741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.374029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.374355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.411117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:39.412353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:39.426672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1190/2380] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:39.427968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.442878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7824140-6e1c-4693-abeb-31ce1fcbc0de None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.448882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.449528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.449810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.450088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.451494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:39.451494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.453377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.471430 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1191/2381] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:39.472856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.492742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3aa6c7-29a0-489c-85e6-53ecc8815739 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.499017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.499808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.500018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.500291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.500980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.502990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.540022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1191/2382] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:39 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:39.541146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.552071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110ae8-1c37-4b11-80c8-b099f5071c39 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.638508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.639381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.639657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.639928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.641033 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.642513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.666835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1192/2383] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:39 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:39.671202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.682558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7248aa6-2b16-46b7-a2ce-9c30be341c4a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.687420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.688022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.688296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.688565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.689247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.691370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.692823 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:39.710629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1192/2384] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:39.711637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.724268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe83c-1c34-490d-a251-f3e02845090d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.729161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.729846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.730197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.730538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.731253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.733036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b8f28-3bab-4597-904a-8cb27e72a2e7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.745281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1193/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:39.749555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.760382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac2029-8911-4d34-a9ec-f8f51367e94e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.765727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.766387 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.766665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.766935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] PATH_INFO: `/v3/projects/cfe1d306716f49be9c1171a3b83d68f3/users/d22dd55b03a44014ba9e52ab79164eeb/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.783593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe1d306716f49be9c1171a3b83d68f3, user_id=d22dd55b03a44014ba9e52ab79164eeb, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.786383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:39.800188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1193/2386] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:39 2022] PUT /identity/v3/projects/cfe1d306716f49be9c1171a3b83d68f3/users/d22dd55b03a44014ba9e52ab79164eeb/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:39.805893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a50a2-744e-45e0-b7dd-a4240622f09b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.806191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a50a2-744e-45e0-b7dd-a4240622f09b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.806473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a50a2-744e-45e0-b7dd-a4240622f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.827703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1c0a50a2-744e-45e0-b7dd-a4240622f09b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:39.828983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c0a50a2-744e-45e0-b7dd-a4240622f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:39.899675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1194/2387] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:39.909812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:39.911648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:39.973670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7620138-b28b-4df6-ab31-24f39dbce758 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:39.978482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:39.979436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:39.980060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:39.980060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:39.990680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:39.992133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:40.003648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1194/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:40.724470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9458fe5-cd3e-4566-aaa1-806260737831 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:40.725471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9458fe5-cd3e-4566-aaa1-806260737831 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:40.725814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9458fe5-cd3e-4566-aaa1-806260737831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:40.765576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c9458fe5-cd3e-4566-aaa1-806260737831 None None] MFA Rules not processed for user `d22dd55b03a44014ba9e52ab79164eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:40.767167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c9458fe5-cd3e-4566-aaa1-806260737831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:40.821236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:40.843162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1195/2389] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:40.854685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:40.856487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:40.916725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf412-ebfd-4e9a-aa4a-511d2b49246f service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:40.921138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:40.921750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:40.922155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:40.922434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:40.931298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:40.932809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:40.942204 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1195/2390] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:42.310344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:42.310898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:42.310898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:42.333858 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] MFA Rules not processed for user `d22dd55b03a44014ba9e52ab79164eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:42.335095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:42.343963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1196/2391] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:42.601466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:42.603485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:42.616329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e23b-4a96-498d-9341-11f34ee48922 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:42.621583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:42.622199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:42.622693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:42.623110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:42.624964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:42.635285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:42.637327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7beec91-b07e-494c-a878-c83d25acf937 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:42.646242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1196/2392] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:43.006486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:43.008537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.122204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bd255-5a33-4bbc-a8b8-a5009c14dd25 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:43.127781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:43.128405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:43.128711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:43.129014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:43.130779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.141495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:43.141899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:43.142176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:43.153207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:43.154886 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:43.165822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1197/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:43.166350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:43.168018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.242906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1197/2394] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:39:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:43.326063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:43.328811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.422624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877845b-53fc-44b9-98ba-6a98cb9ec6c9 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:43.428743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:43.429342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:43.429616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:43.429899 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:43.439319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:43.440953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:43.451205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1198/2395] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:39:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:39:46.444925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:46.448503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:46.467792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:46.642432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248fae5-34fc-401c-81ce-6bd3f8f06cf9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:46.647942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:46.648650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:46.648938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:46.649292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:46.651602 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:46.666265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:46.668052 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188301d0-9d8d-4a83-b264-9689bbe5d9d3 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:46.681037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1198/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:39:53.447248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=73 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:56.317353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:56.331788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5a49f-25c0-41ed-8ea0-ee7ecae45e11 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:56.337491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:56.338106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:56.338423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:56.338801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:56.340541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:56.353155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:56.354775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6a099-f156-47d8-9326-d8970338f250 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:56.366753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1199/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:56.674811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=30 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:59.109405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.110082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.110441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] PATH_INFO: `/` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.112269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1199/2398] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:39:59.119224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52b3d-74c6-410b-8ce6-3cc1d26594be None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.120201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52b3d-74c6-410b-8ce6-3cc1d26594be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.120484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef52b3d-74c6-410b-8ce6-3cc1d26594be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.143276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-7ef52b3d-74c6-410b-8ce6-3cc1d26594be None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:59.144527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7ef52b3d-74c6-410b-8ce6-3cc1d26594be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.154597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1200/2399] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:39:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:59.172679 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:59.174631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.258749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc6739-00a1-4ac0-a907-e0cde20b5b07 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:59.265103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:59.265757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.266042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.266316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.285653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:59.288053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a1578-f712-451a-b4cc-979214670816 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:59.304675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1200/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:59.388816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2186d-c1f2-48f8-abc7-877752f8163e None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.389083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2186d-c1f2-48f8-abc7-877752f8163e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.389381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2186d-c1f2-48f8-abc7-877752f8163e None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.390707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1201/2401] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:39:59.395490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb3f9-5040-490f-b8a4-25275deb484e None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.395779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb3f9-5040-490f-b8a4-25275deb484e None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.396058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb3f9-5040-490f-b8a4-25275deb484e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.425243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-5f4bb3f9-5040-490f-b8a4-25275deb484e None None] MFA Rules not processed for user `a7af5dafacef4215b39c06af7c66c77c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:59.426362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5f4bb3f9-5040-490f-b8a4-25275deb484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.433432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1201/2402] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:39:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:59.446223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2186d-c1f2-48f8-abc7-877752f8163e None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:59.457586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2186d-c1f2-48f8-abc7-877752f8163e None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:59.461453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:59.462136 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.462511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.462856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.470856 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:59.472366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cacce-0c8b-4fb2-b3e4-61a32f5d2bad service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:59.483683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1202/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:59.514440 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf94b7-6bf8-4cb9-be97-1f4a5c2754e3 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.514824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf94b7-6bf8-4cb9-be97-1f4a5c2754e3 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.515202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf94b7-6bf8-4cb9-be97-1f4a5c2754e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.540915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-96cf94b7-6bf8-4cb9-be97-1f4a5c2754e3 None None] MFA Rules not processed for user `d22dd55b03a44014ba9e52ab79164eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:59.542500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-96cf94b7-6bf8-4cb9-be97-1f4a5c2754e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.551084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1202/2404] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:39:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:59.591936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a826ff-1d83-497a-80c9-10bcc50f2c52 None None] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.592547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a826ff-1d83-497a-80c9-10bcc50f2c52 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.593068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a826ff-1d83-497a-80c9-10bcc50f2c52 None None] PATH_INFO: `/` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.594682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1203/2405] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 29 21:39:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:39:59.603657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.604107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.604461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.629890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] MFA Rules not processed for user `0446b5664ae0453a8ccd57dfc562eb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:59.631124 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.642753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1203/2406] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 29 21:39:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:59.655139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a826ff-1d83-497a-80c9-10bcc50f2c52 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:59.668076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a826ff-1d83-497a-80c9-10bcc50f2c52 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:59.671865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:59.672496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.672819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.673091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.685802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:59.687935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9dcb19-4bc8-4039-a76a-4373877e80cd service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:59.700609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1204/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:39:59.820264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:59.835179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb66a6-8435-40f0-ac75-92efc2e40cd5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:59.842788 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:59.844470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:59.846464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:59.846464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:59.848938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:59.867979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:59.868647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:59.878864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1204/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:40:02.035185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:02.035681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:02.035792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:02.057486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:02.059338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:02.068930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1205/2409] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:40:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:02.082267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:02.098840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfb3dc-90f8-44b3-bc32-fb318bae436b service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:02.103829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:02.104687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:02.104687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:02.105490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:02.114409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:02.116001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:02.126106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1205/2410] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:40:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:40:03.629160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:03.647796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca87c3-3b46-40ec-86fe-50a2db53b5d1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:03.656859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:03.657767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:03.658535 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:03.659038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:03.673065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:03.767631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:03.769377 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9feca1-9c29-421d-b4d1-e1a0eea89baa service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:03.790625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1206/2411] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:12.122327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=47 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:12.167013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:12.183118 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347db3a-dcc1-4152-bc09-8784ac6033c0 service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:12.189412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:12.190187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:12.190466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:12.190741 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:12.192534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:12.206483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:12.208268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:12.219744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1206/2412] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:13.775678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=52 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:18.352927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a631890-fe3a-4f49-a2f7-50efd408a6cf None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:18.353845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a631890-fe3a-4f49-a2f7-50efd408a6cf None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:18.354020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a631890-fe3a-4f49-a2f7-50efd408a6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:18.378823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2a631890-fe3a-4f49-a2f7-50efd408a6cf None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:18.382700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2a631890-fe3a-4f49-a2f7-50efd408a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:18.396476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1207/2413] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:40:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:18.411993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:18.433432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae533c2-a1d6-4b7b-abbe-a7f50a152d18 service nova] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:18.439342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:18.439947 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:18.440226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:18.440553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:18.451011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:18.453098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:18.469541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1207/2414] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:40:28.369678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:28.459987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=12 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:31.496449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0e3b-230a-478c-bc24-1ee93ff0c6c7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:31.496975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0e3b-230a-478c-bc24-1ee93ff0c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:31.497091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0e3b-230a-478c-bc24-1ee93ff0c6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:31.522969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c47f0e3b-230a-478c-bc24-1ee93ff0c6c7 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:31.524239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c47f0e3b-230a-478c-bc24-1ee93ff0c6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:31.532245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1208/2415] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:40:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:31.544962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:31.557716 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf984f-6121-48f6-8182-22e56b6e676a service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:31.563348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:31.563964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:31.564242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:31.564535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:31.574028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:31.575622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:31.584319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1208/2416] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:40:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:40:41.516413 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:41.581783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:51.586107 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:51.586940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:51.587096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:51.619654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:51.622004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:51.636098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1209/2417] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:40:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:51.655528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:51.667701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792fd84-6e1d-47bd-9f7d-0afbce94573f service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:51.678272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:51.680772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:51.680772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:51.680772 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:51.697662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:51.701882 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:51.719053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1209/2418] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:40:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:40:54.065197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:54.067501 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.095421 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0482d3f-9cd3-42ba-8996-24a918184190 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:54.104316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:54.105404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.105906 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.106270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.109237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.128232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:54.131802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:54.150276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:54.153672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1210/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:54.162797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21de7b-4054-4941-9048-f2ccfd5a823b service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:54.175802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:54.177767 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.178361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.178798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] PATH_INFO: `/v3/users/d22dd55b03a44014ba9e52ab79164eeb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.181036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22dd55b03a44014ba9e52ab79164eeb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:54.184924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:54.259712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22dd55b03a44014ba9e52ab79164eeb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.273332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22dd55b03a44014ba9e52ab79164eeb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.284092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22dd55b03a44014ba9e52ab79164eeb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.285378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22dd55b03a44014ba9e52ab79164eeb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.286377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22dd55b03a44014ba9e52ab79164eeb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.315274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1210/2420] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:40:54 2022] DELETE /identity/v3/users/d22dd55b03a44014ba9e52ab79164eeb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:40:54.595346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:54.610999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:54.612531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.628895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291e09-121e-48c1-b8fe-f89fec210c2d service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:54.706452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e014960-42f3-477f-b4af-ccd758387287 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:54.712477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:54.713424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.713916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.714430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.718449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.730684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:54.731732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.732185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.732685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] PATH_INFO: `/v3/projects/cfe1d306716f49be9c1171a3b83d68f3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.735471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe1d306716f49be9c1171a3b83d68f3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:54.740139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:54.814354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe1d306716f49be9c1171a3b83d68f3', 'request_id': 'req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.817576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:54.820332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff42848-5616-4ede-9776-b580ed2a811d service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:54.824260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe1d306716f49be9c1171a3b83d68f3', 'request_id': 'req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.835447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe1d306716f49be9c1171a3b83d68f3', 'request_id': 'req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.838282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1211/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:40:54.853126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] The token cache is being invalidate because project cfe1d306716f49be9c1171a3b83d68f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:40:54.853559 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.854401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-3a69678f-e7c2-47ea-9112-3ea9c5577f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.857166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1211/2422] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:40:54 2022] DELETE /identity/v3/projects/cfe1d306716f49be9c1171a3b83d68f3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:40:54.865223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.865749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.867035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.891924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:54.894380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.902306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:54.989261 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1212/2423] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:54.998924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.999209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.999490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.023935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:55.025964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:55.037765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1212/2424] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:40:55.044322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.063856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49c387-82dd-48b8-93a7-a1806917d857 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.073207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.074286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.075017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.075544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.076793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:40:55.077478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.080979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.101031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1213/2425] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:40:55.107209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.124371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017f8b-7bb2-4216-90ee-e6960f4354c5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.132233 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.133314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.133861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.134341 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.135563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.139815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.193173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1213/2426] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:40:55 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:40:55.199789 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.218166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768129f2-db61-466d-9bad-3a07716f24f0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.299590 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.300348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.300811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.301227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.302132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.304952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.330864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1214/2427] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:40:55 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:40:55.335654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.353807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09ed7-4a0b-4e47-b04c-f7d7867bd564 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.360342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.360720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.360882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.361223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.362116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.365649 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.385106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1214/2428] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:40:55.392675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.412678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8155-0d7e-42ff-93c6-7cba251f3f2f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.420627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.421671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.422243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.422773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.424005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.426726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b60b59-d707-4e76-89fb-3f99ebe50d21 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.445299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1215/2429] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:40:55.449986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.465707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f73ce-a5d6-4b7f-a0db-a6c0d70c06a6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.472910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.474080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.475074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.475584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] PATH_INFO: `/v3/projects/76615167df444acdbd035a00947f8246/users/62d01437f08d46a0be679c5f9e7ae67b/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.500420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] RBAC: Authorizing `identity:create_grant(project_id=76615167df444acdbd035a00947f8246, user_id=62d01437f08d46a0be679c5f9e7ae67b, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.503146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.523119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1215/2430] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:40:55 2022] PUT /identity/v3/projects/76615167df444acdbd035a00947f8246/users/62d01437f08d46a0be679c5f9e7ae67b/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:40:55.539427 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.539915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.540370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.585809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] MFA Rules not processed for user `62d01437f08d46a0be679c5f9e7ae67b`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:40:55.588100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:55.720188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1216/2431] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:40:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:40:55.744972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.748306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:55.757956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:55.760733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:55.770269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:55.771929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:55.869552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10113a-7ece-4f0d-834a-59718805cd8d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.878868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.879984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.880171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638493f2-93b8-4b34-9a26-37d4f7ce30af admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:55.880690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.881126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.884933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:55.886745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:55.888067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:55.888572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:55.889123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:55.906641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.909309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:55.924647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1216/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:55.986963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:55.988441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:56.001892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1217/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:01.134196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:01.139871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:01.164869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247a3a-b6e3-470b-87c3-56c942f2729e service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:01.176224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:01.177099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:01.177476 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:01.177835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] PATH_INFO: `/v3/users/89b272427c6444b49293e5046c686ffc` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:01.179496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b272427c6444b49293e5046c686ffc)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:01.182171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:01.302676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b272427c6444b49293e5046c686ffc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:01.329974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b272427c6444b49293e5046c686ffc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:01.369759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b272427c6444b49293e5046c686ffc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:01.375031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b272427c6444b49293e5046c686ffc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:01.416971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-323aa5bb-24e2-4f57-947a-7cb1b97c5ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b272427c6444b49293e5046c686ffc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:01.436218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1217/2434] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:01 2022] DELETE /identity/v3/users/89b272427c6444b49293e5046c686ffc => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:02.274057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:02.306221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a4ae4-4f9c-465d-bd17-cd86b1987893 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:02.497711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:02.498458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:02.498732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:02.499071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] PATH_INFO: `/v3/projects/5220f1dfa6a74754bd08f925c85847b2` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:02.500654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5220f1dfa6a74754bd08f925c85847b2)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:02.513133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:02.588774 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5220f1dfa6a74754bd08f925c85847b2', 'request_id': 'req-661092d5-4ea9-4e75-aac1-b9e86e7eee70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:02.600223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5220f1dfa6a74754bd08f925c85847b2', 'request_id': 'req-661092d5-4ea9-4e75-aac1-b9e86e7eee70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:02.603022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:02.618045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5220f1dfa6a74754bd08f925c85847b2', 'request_id': 'req-661092d5-4ea9-4e75-aac1-b9e86e7eee70'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:02.647268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] The token cache is being invalidate because project 5220f1dfa6a74754bd08f925c85847b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:02.647740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:02.648803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-661092d5-4ea9-4e75-aac1-b9e86e7eee70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:02.651396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1218/2435] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:02 2022] DELETE /identity/v3/projects/5220f1dfa6a74754bd08f925c85847b2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:02.656098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:02.657624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:02.658101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:02.681115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:02.682929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:02.783154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1218/2436] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:02.793444 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971da89-0237-440f-9db8-e8374c808872 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:02.793876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971da89-0237-440f-9db8-e8374c808872 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:02.794251 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971da89-0237-440f-9db8-e8374c808872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:02.821382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-c971da89-0237-440f-9db8-e8374c808872 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:02.823631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c971da89-0237-440f-9db8-e8374c808872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:02.843313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1219/2437] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:02.845735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:02.874660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6c5ba-c822-4953-a4b4-613d9fb280ae None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:02.881425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:02.882536 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:02.882911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:02.883344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:02.884128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:02.884706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:02.887758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:02.907142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1219/2438] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:02.912878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971da89-0237-440f-9db8-e8374c808872 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:02.927454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971da89-0237-440f-9db8-e8374c808872 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:02.934194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:02.934966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:02.935351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:02.935681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:02.936450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:02.938826 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:02.976431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:02.979179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:02.995495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1220/2439] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:02 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:02.996696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.005050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d739a-832f-41e0-9479-ce5dd46b87d9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.019960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:03.116401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48830b-2c32-48fa-a921-990c7c88871b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.124305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.125305 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.125764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.126045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.126232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.126875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.127155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.127460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.127784 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] PATH_INFO: `/v3/users/6f0d97b196f44ec985db30f86d4ad3f4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.129281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f0d97b196f44ec985db30f86d4ad3f4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.132844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.133933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.187797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1221/2440] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:02 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:03.193885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.207868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970854a-d4f1-4cee-8982-8583eecd1966 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.214448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.215516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.215719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.215926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.216660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.218288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.236158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1222/2441] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:03.237744 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f0d97b196f44ec985db30f86d4ad3f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.241982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.251170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f0d97b196f44ec985db30f86d4ad3f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.255719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac197d-efad-469e-a57f-ad8546246675 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.263659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.264939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f0d97b196f44ec985db30f86d4ad3f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.265157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.267842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.267842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.267842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.270866 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f0d97b196f44ec985db30f86d4ad3f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.281301 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.292279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-2ca1d93e-3b14-4575-8060-42905f2cc3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f0d97b196f44ec985db30f86d4ad3f4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.295853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1223/2442] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:03.301935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.304627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:03.311798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1220/2443] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:02 2022] DELETE /identity/v3/users/6f0d97b196f44ec985db30f86d4ad3f4 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:03.340871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec3d2e-b7aa-47d2-8bbf-6b927f9cfd89 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.422971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.423555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.423832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.424291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] PATH_INFO: `/v3/projects/025ca4f33e7146a5a7ade9bd67c9ed1e/users/d8070f3213574b77bc872698015993d1/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.446077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=025ca4f33e7146a5a7ade9bd67c9ed1e, user_id=d8070f3213574b77bc872698015993d1, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.449298 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.471211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1224/2444] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:03 2022] PUT /identity/v3/projects/025ca4f33e7146a5a7ade9bd67c9ed1e/users/d8070f3213574b77bc872698015993d1/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:03.485830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.487573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.488145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.547689 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] MFA Rules not processed for user `d8070f3213574b77bc872698015993d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:03.552667 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:03.679922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.683371 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:03.714686 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:03.765210 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1221/2445] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:03.766639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.798657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:03.862713 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6062f-5b1f-40f8-90ef-3f245ffd0f0c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.871465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.872138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.872410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.872738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] PATH_INFO: `/v3/projects/7d9928c37b2b46f5abd0ac1ba1c3ca92` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.873933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9928c37b2b46f5abd0ac1ba1c3ca92)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.876193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.904634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4a374-e015-4a78-9c43-dcc484befcd5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:03.916031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:03.917605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:03.918073 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:03.918586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:03.919794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:03.924871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:03.970182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9928c37b2b46f5abd0ac1ba1c3ca92', 'request_id': 'req-762173e2-ab8e-4d2b-ba73-82503f0c5d75'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.983468 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9928c37b2b46f5abd0ac1ba1c3ca92', 'request_id': 'req-762173e2-ab8e-4d2b-ba73-82503f0c5d75'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:03.983768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1222/2446] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:03 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:03.984647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:03.994084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9928c37b2b46f5abd0ac1ba1c3ca92', 'request_id': 'req-762173e2-ab8e-4d2b-ba73-82503f0c5d75'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:04.008808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] The token cache is being invalidate because project 7d9928c37b2b46f5abd0ac1ba1c3ca92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:04.008808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:04.010446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-762173e2-ab8e-4d2b-ba73-82503f0c5d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:04.015420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1225/2447] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:03 2022] DELETE /identity/v3/projects/7d9928c37b2b46f5abd0ac1ba1c3ca92 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:04.023665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c171a5-b468-43eb-a869-6e7acc61c54f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.024682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c171a5-b468-43eb-a869-6e7acc61c54f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.025337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c171a5-b468-43eb-a869-6e7acc61c54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.055195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-63c171a5-b468-43eb-a869-6e7acc61c54f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:04.057098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-63c171a5-b468-43eb-a869-6e7acc61c54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:04.122481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a9584-0baa-4e34-a977-f9ae034faf07 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.133329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.134357 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.134822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.135282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.137372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.141516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.163169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1226/2448] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:04.172133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.172500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.172946 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.192250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1223/2449] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:03 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:04.196832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.199605 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:04.206028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:04.207632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:04.219389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f2139-57c8-45c7-ae45-80bcd2d617e2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.226935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1227/2450] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:04.228616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.229012 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.229961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.230447 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.230923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.232865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.235207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.238858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac4223-8d39-43e2-ae4d-d2765e4fd070 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.245895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.247248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.247248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.247248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.248013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:04.248325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.251031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.255712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1224/2451] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:04.260301 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.276981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b97337-1f3d-4f0e-afc2-b92bbf22c7b2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.282034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1228/2452] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:04.283190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.286864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.287924 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.288321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.288723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] PATH_INFO: `/v3/projects/3afa691f89a640a688795a3c1005abb4/users/463256cbae794dfcb1ec147cc88d0d0b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.297356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ffe78-b0d3-487a-a34e-7bd58b824929 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.303867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.304790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.305128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.305660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.308318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.313633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.318516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afa691f89a640a688795a3c1005abb4, user_id=463256cbae794dfcb1ec147cc88d0d0b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.328805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.340799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:04.365527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1225/2453] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:04 2022] PUT /identity/v3/projects/3afa691f89a640a688795a3c1005abb4/users/463256cbae794dfcb1ec147cc88d0d0b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:04.370178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.387973 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1229/2454] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:04 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:04.389514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.525974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:04.545531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb252af-461f-463b-b129-c679864a24a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.553673 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9250f-49fa-4b15-a71f-f1c3f4130b1f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.558402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.560005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.561060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.561667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.562192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.562701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.563195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.563677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.563855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.564897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.567237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.569064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.592736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1226/2455] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:04.599888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.607888 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1230/2456] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:04 2022] POST /identity/v3/users => generated 303 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:04.619688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.626921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb1018-07ec-4592-98e3-e7f2d202dd96 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.640637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7a639-4355-498b-b22a-0b7cc263738f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.642910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.643610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.643925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.644189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] PATH_INFO: `/v3/projects/3afa691f89a640a688795a3c1005abb4/users/463256cbae794dfcb1ec147cc88d0d0b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.646760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afa691f89a640a688795a3c1005abb4, user_id=463256cbae794dfcb1ec147cc88d0d0b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.649464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a093f8-7f7d-4a73-a099-51edcc45503a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.649786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.650882 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.651650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.652185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.653381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.656447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.662847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1227/2457] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:04 2022] PUT /identity/v3/projects/3afa691f89a640a688795a3c1005abb4/users/463256cbae794dfcb1ec147cc88d0d0b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:04.682064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.682625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.682966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.683612 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1231/2458] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:04.685386 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.707337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4032-61e3-4a47-981b-560f47183d12 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.746698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] MFA Rules not processed for user `463256cbae794dfcb1ec147cc88d0d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:04.748627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:04.886426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.887944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.888674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.889170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:04.890346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:04.894352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:04.917296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1232/2459] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:04.925183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.928387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1228/2460] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:04.944984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbabf0-03a1-4e3d-886f-71ae20520c6a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:04.946892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:04.948355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:04.973432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:04.974350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:04.975755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:04.975755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] PATH_INFO: `/v3/projects/ea725b9278fb4042ba2ed444502b6795/users/8499745e3c2044949333952f08562e5f/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:05.030630 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea725b9278fb4042ba2ed444502b6795, user_id=8499745e3c2044949333952f08562e5f, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:05.034354 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dc08d-ea02-4fbd-aece-a04413e878a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:05.072995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1233/2461] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:04 2022] PUT /identity/v3/projects/ea725b9278fb4042ba2ed444502b6795/users/8499745e3c2044949333952f08562e5f/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:05.078886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:05.079495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:05.080031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:05.099098 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb672aa-75bd-47bf-af04-d9abdaa33f21 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:05.117810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:05.119072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:05.222091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:05.222091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:05.222091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:05.222091 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:05.232576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:05.299725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1234/2462] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:05.314611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:05.316816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:05.323219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:05.456970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:05.462100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:05.505782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1229/2463] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:05.569865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950d75a-41a8-418f-ba7f-142124460480 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:05.576987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:05.577685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:05.578071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:05.578815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:05.608657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:05.612615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e930a76-a5b2-45fc-a161-9ce5899e9925 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:05.635489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1235/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:05.704793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:05.707287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:05.727383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1be3f-cf5d-457e-817c-178341d62bd1 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:05.735568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:05.737632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:05.737632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:05.737632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:05.740781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:05.875499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:05.877095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:05.892823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1230/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:06.705344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:06.706243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:06.706521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:06.743056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] MFA Rules not processed for user `8499745e3c2044949333952f08562e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:06.745089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:06.890425 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1236/2466] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:06.898643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:06.923707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f4373-7cd7-4ce2-81a4-b02b629a8d42 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:06.936573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:06.937701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:06.938168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:06.938646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:06.940311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:06.943972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.018157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1231/2467] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:06 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:07.019758 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:07.037603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db20f7-3ef9-473e-aef3-0eec785e26c1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:07.173120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:07.173808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:07.174086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:07.174362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:07.175011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:07.177777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.220363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1237/2468] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:07 2022] POST /identity/v3/users => generated 309 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:07.224718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:07.240942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168bf5-4299-417c-b25a-9b4b3b8f78f9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:07.249065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:07.250214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:07.251905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:07.252757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:07.254392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:07.257670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.299056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1232/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:07.302790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:07.318269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ef340-09bf-4d58-b4a5-59eb4a58e5b1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:07.324570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:07.325342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:07.325637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:07.325916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] PATH_INFO: `/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b/users/7ecd0ee3c693488cba0015417f2cd80b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:07.330976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:07.361775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d86d0bbd4984d7e8e715dc565f44c7b, user_id=7ecd0ee3c693488cba0015417f2cd80b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:07.368211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.397428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1238/2470] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:07 2022] PUT /identity/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b/users/7ecd0ee3c693488cba0015417f2cd80b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:07.402075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:07.422568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288cbe7-1cf8-433a-9570-21a7133c8b3b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:07.561345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:07.562340 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:07.562854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:07.563239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:07.564298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:07.568163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07409b00-4dfc-4085-aeb9-9904fef6aaa4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.585659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1233/2471] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:07.591420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:07.615061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57e20c-a722-40f9-b6c4-ff180a91c526 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:07.623685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:07.625181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:07.625891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:07.626418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] PATH_INFO: `/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b/users/7ecd0ee3c693488cba0015417f2cd80b/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:07.630809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d86d0bbd4984d7e8e715dc565f44c7b, user_id=7ecd0ee3c693488cba0015417f2cd80b, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:07.634035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:07.649884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1239/2472] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:07 2022] PUT /identity/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b/users/7ecd0ee3c693488cba0015417f2cd80b/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:08.565436 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768d75-8f91-4c52-a548-7c07c1cb7e42 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:08.566378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768d75-8f91-4c52-a548-7c07c1cb7e42 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:08.566933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a768d75-8f91-4c52-a548-7c07c1cb7e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:08.627199 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6a768d75-8f91-4c52-a548-7c07c1cb7e42 None None] MFA Rules not processed for user `7ecd0ee3c693488cba0015417f2cd80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:08.629491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6a768d75-8f91-4c52-a548-7c07c1cb7e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:08.683445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:08.789090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1234/2473] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:08 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:09.211683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:09.214390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:09.309022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bfef9-464e-479d-b057-2bb8e4d68835 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:09.317618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:09.318622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:09.319289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:09.319763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] PATH_INFO: `/v3/users/62d01437f08d46a0be679c5f9e7ae67b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:09.321528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d01437f08d46a0be679c5f9e7ae67b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:09.324479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:09.402786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d01437f08d46a0be679c5f9e7ae67b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:09.431476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d01437f08d46a0be679c5f9e7ae67b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:09.440286 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d01437f08d46a0be679c5f9e7ae67b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:09.440977 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d01437f08d46a0be679c5f9e7ae67b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:09.441617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d01437f08d46a0be679c5f9e7ae67b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:09.458564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1240/2474] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:09 2022] DELETE /identity/v3/users/62d01437f08d46a0be679c5f9e7ae67b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:09.463085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:09.463380 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:09.463659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:09.504219 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:09.506705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:09.617712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1235/2475] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:09.628134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:09.629981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:09.712039 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50689fd-4f04-499a-a675-c4a4084930d8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:09.715729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:09.718028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:09.718028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:09.718028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:09.725162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:09.731573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:09.734207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:09.750389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1241/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:09.961918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:09.963361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:09.976303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103af4fa-eb9a-4dab-b6a8-6ec3df054896 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:09.980275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:09.980983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:09.981314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:09.982045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:09.983564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:09.997583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:09.999019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:10.012072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1236/2477] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:10.289223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:10.291351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:10.310859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed8d0-1ccb-4217-bf2c-0150d23951fb service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:10.317668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:10.318340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:10.318817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:10.319102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] PATH_INFO: `/v3/users/8499745e3c2044949333952f08562e5f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:10.320349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8499745e3c2044949333952f08562e5f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:10.322979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:10.403738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8499745e3c2044949333952f08562e5f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.425334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8499745e3c2044949333952f08562e5f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.437294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8499745e3c2044949333952f08562e5f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.442035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8499745e3c2044949333952f08562e5f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.443854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8499745e3c2044949333952f08562e5f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.467093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1242/2478] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:10 2022] DELETE /identity/v3/users/8499745e3c2044949333952f08562e5f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:10.614057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:10.620115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:10.642696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd26c3-ef52-4878-954a-fbd8e0c693f9 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:10.745018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:10.746601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:10.756574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:10.763927 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:10.764408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:10.764883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] PATH_INFO: `/v3/projects/76615167df444acdbd035a00947f8246` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:10.768467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=76615167df444acdbd035a00947f8246)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:10.771910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:10.843390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:10.846078 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76615167df444acdbd035a00947f8246', 'request_id': 'req-23ba6278-a73f-4cff-ba60-f97ec748c5cd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.857514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76615167df444acdbd035a00947f8246', 'request_id': 'req-23ba6278-a73f-4cff-ba60-f97ec748c5cd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.872747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8c9c0-a0d1-488d-9977-2424aa7bc486 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:10.875171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76615167df444acdbd035a00947f8246', 'request_id': 'req-23ba6278-a73f-4cff-ba60-f97ec748c5cd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.891913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] The token cache is being invalidate because project 76615167df444acdbd035a00947f8246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:10.892202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.892975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-23ba6278-a73f-4cff-ba60-f97ec748c5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:10.896363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1237/2479] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:10 2022] DELETE /identity/v3/projects/76615167df444acdbd035a00947f8246 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:10.911232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7432240-61fe-4aa4-9271-67e9432f2d5d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:10.911517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7432240-61fe-4aa4-9271-67e9432f2d5d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:10.911845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7432240-61fe-4aa4-9271-67e9432f2d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:10.948204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c7432240-61fe-4aa4-9271-67e9432f2d5d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:10.950446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c7432240-61fe-4aa4-9271-67e9432f2d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:10.972875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:10.974172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:10.974478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:10.974748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] PATH_INFO: `/v3/projects/ea725b9278fb4042ba2ed444502b6795` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:10.975940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea725b9278fb4042ba2ed444502b6795)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:10.978443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.007814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:11.060995 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1238/2480] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:11.075963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.077106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.077773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.077952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea725b9278fb4042ba2ed444502b6795', 'request_id': 'req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.092638 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea725b9278fb4042ba2ed444502b6795', 'request_id': 'req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.105001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea725b9278fb4042ba2ed444502b6795', 'request_id': 'req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.116262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:11.118975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:11.131914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] The token cache is being invalidate because project ea725b9278fb4042ba2ed444502b6795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:11.132215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.133793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.138008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1243/2481] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:10 2022] DELETE /identity/v3/projects/ea725b9278fb4042ba2ed444502b6795 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:11.142949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.144665 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:11.233843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1239/2482] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:11.237953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.254145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407a74b-13c8-4a48-bfaf-2f70ec589a05 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.262242 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.263248 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.263731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.264182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.265451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:11.265952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.269532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.271111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecffe6-8ae8-401b-93c1-59bde05e90c7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.280437 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.281493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.282036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.283100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] PATH_INFO: `/v3/users/7ecd0ee3c693488cba0015417f2cd80b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.284918 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecd0ee3c693488cba0015417f2cd80b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.289366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.295794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1240/2483] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:11.297387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.319815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa4977-eaf4-457f-8b56-c40ad7595d60 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.330823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.332088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.333274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.333822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.335675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.338857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.369076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecd0ee3c693488cba0015417f2cd80b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.398041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1241/2484] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:11 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:11.398736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecd0ee3c693488cba0015417f2cd80b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.399388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.411992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecd0ee3c693488cba0015417f2cd80b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.414000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecd0ee3c693488cba0015417f2cd80b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.415986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecd0ee3c693488cba0015417f2cd80b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:11.446257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1244/2485] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:11 2022] DELETE /identity/v3/users/7ecd0ee3c693488cba0015417f2cd80b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:11.474934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:11.520970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a54566-6bea-465d-9ccc-9467b37abd93 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.644228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.646041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.646541 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.647036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.648227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.653087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.689372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1242/2486] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:11 2022] POST /identity/v3/users => generated 286 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:11.696557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.699274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:11.727012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b95a2-09de-44b7-8456-9bbeee51fea1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.737250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.738415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.738849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.739446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.740479 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.743093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.765634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1245/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:11.787368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.806826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a5071-8ea9-46ad-ae79-ce220cfe4dda admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.816247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.817705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.818000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.818401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.819642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.822360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.837853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1243/2488] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:11.842211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.866053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3064e3-521d-4311-9a75-a333fd156894 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:11.873431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:11.876648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:11.876648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:11.876648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] PATH_INFO: `/v3/projects/dbf312281cdc4ad4925d1815c3ac7881/users/b8ab0d2a5e1b4b4ab4378bd89d2db123/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:11.900136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:11.914870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf312281cdc4ad4925d1815c3ac7881, user_id=b8ab0d2a5e1b4b4ab4378bd89d2db123, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:11.920796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68302f6d-24d6-4037-ac07-b04669c4d4ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:11.939137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:11.943654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:11.961678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1246/2489] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:11 2022] PUT /identity/v3/projects/dbf312281cdc4ad4925d1815c3ac7881/users/b8ab0d2a5e1b4b4ab4378bd89d2db123/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:12.015771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b1176-8cea-4a1c-8b0f-cb95a526a6b4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:12.044613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.044937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.045347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.086524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] MFA Rules not processed for user `b8ab0d2a5e1b4b4ab4378bd89d2db123`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:12.087863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:12.117495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:12.119522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.121007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.121566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] PATH_INFO: `/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.123679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d86d0bbd4984d7e8e715dc565f44c7b)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:12.127971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:12.216266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d86d0bbd4984d7e8e715dc565f44c7b', 'request_id': 'req-d9b76dca-2713-4882-b2c0-c1a97d400626'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:12.220650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1247/2490] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:12.228008 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d86d0bbd4984d7e8e715dc565f44c7b', 'request_id': 'req-d9b76dca-2713-4882-b2c0-c1a97d400626'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:12.236567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:12.238339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:12.244300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d86d0bbd4984d7e8e715dc565f44c7b', 'request_id': 'req-d9b76dca-2713-4882-b2c0-c1a97d400626'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:12.269198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:12.286675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] The token cache is being invalidate because project 0d86d0bbd4984d7e8e715dc565f44c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:12.288207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:12.289973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d9b76dca-2713-4882-b2c0-c1a97d400626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:12.292123 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1244/2491] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:11 2022] DELETE /identity/v3/projects/0d86d0bbd4984d7e8e715dc565f44c7b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:12.298985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542ec83-1f87-45fb-8a73-be701ae2a86d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.299310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542ec83-1f87-45fb-8a73-be701ae2a86d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.299742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542ec83-1f87-45fb-8a73-be701ae2a86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.334891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6542ec83-1f87-45fb-8a73-be701ae2a86d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:12.336601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6542ec83-1f87-45fb-8a73-be701ae2a86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:12.342381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a3ac-8f0a-478a-97a9-8eccb5d91a91 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:12.422147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:12.423669 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.425090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.425401 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.428482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:12.446326 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1245/2492] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:12.453839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.454752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.455910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.490371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:12.492682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:12.527208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:12.530217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1246/2493] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:12.538593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:12.565237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:12.567525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:12.571648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afe872-8284-4b01-bef8-7237445f0940 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:12.579093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1248/2494] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:12.580507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:12.582044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.582479 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.583081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.584444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:12.585246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:12.588775 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:12.621905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1247/2495] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:41:12.628150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:12.651384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e2c3-2329-4d5d-bc24-2b4655b94fcf service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:12.664501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:12.665825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:12.667374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:12.667374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:12.667374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:12.669847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:12.757497 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1249/2496] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:12 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:12.763368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:12.805880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052f2d-c31f-433e-8751-29fa738e0519 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:13.101023 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:13.101936 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:13.102514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:13.102913 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:13.104159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:13.124114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:13.214231 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1248/2497] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:12 2022] POST /identity/v3/users => generated 287 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:13.217915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:13.288554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec9cf-10c3-44de-8c97-e81803a856e1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:13.336260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:13.339062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:13.339528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:13.341770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:13.350065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:13.360461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:13.437189 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:13.437449 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1250/2498] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:13.460698 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e8856-4f1a-4991-803e-d11ab0beead5 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:13.469964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:13.471146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:13.471888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:13.472686 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:13.474272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:13.477701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db78d0-68e8-40e1-935d-3c09a8a0d76c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:13.504306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1249/2499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:13.507660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:13.535902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25245-4d29-4465-b278-f1a6decc5fde admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:13.558442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:13.561684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:13.565762 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:13.566123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] PATH_INFO: `/v3/projects/f63f7676490e42029f07413387b9f08a/users/b5ada17b384640ab968610cacb8b7f79/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:13.614370 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63f7676490e42029f07413387b9f08a, user_id=b5ada17b384640ab968610cacb8b7f79, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:13.624128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:13.681138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1251/2500] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:13 2022] PUT /identity/v3/projects/f63f7676490e42029f07413387b9f08a/users/b5ada17b384640ab968610cacb8b7f79/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:13.700524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:13.700954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:13.701435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:13.757300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] MFA Rules not processed for user `b5ada17b384640ab968610cacb8b7f79`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:13.759382 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:13.873797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:13.924456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1250/2501] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:13.929053 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.050525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b51008-8bcb-46bd-8668-09f17588ff89 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.067264 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.068515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.069180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.069796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.070952 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.075068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.128714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1252/2502] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:13 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:14.135637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.151116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e380d-d50a-454c-b162-537c2eb24b12 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.266655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.267708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.268151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.268836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.269846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.274194 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.321446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1251/2503] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:14 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:14.325828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.345084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd6c9-e619-4790-b865-dd2890023050 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.354185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.355606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.355606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.355606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.356335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.358483 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.373234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1253/2504] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:14.377948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.393370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9704c26-75f7-45f4-97b3-06afa5dba974 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.405670 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.406524 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.406939 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.407250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] PATH_INFO: `/v3/projects/361a393f005e40679385fe26d5967781/users/a44748ca35584878984af98fc37007f7/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.436149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361a393f005e40679385fe26d5967781, user_id=a44748ca35584878984af98fc37007f7, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.439360 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7090d03-f3bb-4071-9d70-77cf79eeae29 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.465875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1252/2505] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:14 2022] PUT /identity/v3/projects/361a393f005e40679385fe26d5967781/users/a44748ca35584878984af98fc37007f7/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:14.467715 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.479817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882d3f2-93d0-4de8-9f9f-6f24c49c6e19 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.583615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.584310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.585125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.585566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.585688 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.585924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.586203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.587036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.588889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.595609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:14.611210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1254/2506] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:14.617238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.619600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] MFA Rules not processed for user `b8ab0d2a5e1b4b4ab4378bd89d2db123`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:14.621070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:14.643207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0325ae4-29b0-42cf-aea6-f6c156d93610 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:14.653555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:14.654146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.654428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.654708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] PATH_INFO: `/v3/projects/361a393f005e40679385fe26d5967781/users/a44748ca35584878984af98fc37007f7/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.659718 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=361a393f005e40679385fe26d5967781, user_id=a44748ca35584878984af98fc37007f7, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:14.662812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71e58-46c6-4bb2-a9f4-65a1b352a78f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:14.676168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1255/2507] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:14 2022] PUT /identity/v3/projects/361a393f005e40679385fe26d5967781/users/a44748ca35584878984af98fc37007f7/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:14.693996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:14.694315 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:14.694548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:14.749567 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1253/2508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:14.776524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] MFA Rules not processed for user `a44748ca35584878984af98fc37007f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:14.780999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:14.903522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1256/2509] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:14.915992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:14.918680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:15.101683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d65ea-713e-44bc-b931-ead9ce1d8d9c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:15.125096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:15.249806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:15.251628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:15.252850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:15.253459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:15.280980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:15.282761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:15.300358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1254/2510] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:15.388006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:15.393516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:15.509509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:15.525446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606f00-91e6-4a2a-9d8f-48b7b2ef7485 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:15.543996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:15.544984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:15.545699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:15.546302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:15.589294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f7a26-8959-4b46-b9c9-e2552f380226 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:15.596254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:15.598828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:15.599457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:15.599969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] PATH_INFO: `/v3/users/d8070f3213574b77bc872698015993d1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:15.603736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8070f3213574b77bc872698015993d1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:15.609400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:15.665966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:15.670467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e7eb5-adb1-4def-8401-034c2d975073 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:15.694863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1255/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:15.729643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8070f3213574b77bc872698015993d1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:15.758865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8070f3213574b77bc872698015993d1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:15.790231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8070f3213574b77bc872698015993d1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:15.791618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8070f3213574b77bc872698015993d1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:15.793328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8070f3213574b77bc872698015993d1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:15.799803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:15.824495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1257/2512] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:15 2022] DELETE /identity/v3/users/d8070f3213574b77bc872698015993d1 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:15.829409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:15.829983 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:15.830131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:15.858971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:15.860277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.001607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1256/2513] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:16.014445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:16.016561 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.122352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b9c14-336a-475c-8396-f6c537a2042f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:16.128836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:16.130135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:16.130560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:16.131009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:16.148932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:16.152214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:16.172790 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1258/2514] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:16.635630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:16.637849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.641567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:16.643405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.670986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118369ee-3e67-47e1-a00a-a218f18d4820 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:16.703337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:16.704490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:16.705624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:16.706134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] PATH_INFO: `/v3/projects/025ca4f33e7146a5a7ade9bd67c9ed1e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:16.709331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=025ca4f33e7146a5a7ade9bd67c9ed1e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:16.712989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:16.751262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:16.795034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4e1ff-a070-497f-ac8e-82809e580399 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:16.806243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:16.807582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:16.808063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:16.808308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:16.812452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.837749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '025ca4f33e7146a5a7ade9bd67c9ed1e', 'request_id': 'req-0f73c719-c9db-4d88-a564-c272f9c22e06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:16.838063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:16.841083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:16.854017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '025ca4f33e7146a5a7ade9bd67c9ed1e', 'request_id': 'req-0f73c719-c9db-4d88-a564-c272f9c22e06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:16.861951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1257/2515] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:16.865445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:16.869050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:16.876717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '025ca4f33e7146a5a7ade9bd67c9ed1e', 'request_id': 'req-0f73c719-c9db-4d88-a564-c272f9c22e06'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:16.894359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c234e-8a41-473d-aaef-b0bd770ffc20 service placement] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:16.912864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] The token cache is being invalidate because project 025ca4f33e7146a5a7ade9bd67c9ed1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:16.913462 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:16.916937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:16.921278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1259/2516] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:16 2022] DELETE /identity/v3/projects/025ca4f33e7146a5a7ade9bd67c9ed1e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:16.926499 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:16.929400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:17.000014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:17.000996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:17.001533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:17.002497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] PATH_INFO: `/v3/users/55a5c6359109440fbb66d985fafc24bf` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:17.005212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a5c6359109440fbb66d985fafc24bf)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:17.007896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:17.113190 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a5c6359109440fbb66d985fafc24bf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.113622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73c719-c9db-4d88-a564-c272f9c22e06 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:17.121352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:17.122073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:17.122455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:17.122832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] PATH_INFO: `/v3/users/463256cbae794dfcb1ec147cc88d0d0b` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:17.124213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] RBAC: Authorizing `identity:delete_user(user_id=463256cbae794dfcb1ec147cc88d0d0b)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:17.127648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:17.139404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a5c6359109440fbb66d985fafc24bf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.167491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a5c6359109440fbb66d985fafc24bf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.168935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a5c6359109440fbb66d985fafc24bf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.193315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a5c6359109440fbb66d985fafc24bf'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.194367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:17.204700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1258/2517] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:16 2022] DELETE /identity/v3/users/55a5c6359109440fbb66d985fafc24bf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:17.265988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463256cbae794dfcb1ec147cc88d0d0b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.286510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463256cbae794dfcb1ec147cc88d0d0b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.318970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463256cbae794dfcb1ec147cc88d0d0b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.319539 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463256cbae794dfcb1ec147cc88d0d0b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.320827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463256cbae794dfcb1ec147cc88d0d0b'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:17.360117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1260/2518] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:16 2022] DELETE /identity/v3/users/463256cbae794dfcb1ec147cc88d0d0b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:17.601555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:17.618802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:17.920834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f94210-ea1a-4f5c-988d-10819104b0cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:17.941338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:18.050648 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.052075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.052557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.053216 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] PATH_INFO: `/v3/projects/09369e28336d4f49a60ce03c3a3009ac` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.056070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09369e28336d4f49a60ce03c3a3009ac)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.060747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.099874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.102993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:18.136079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e023-5f79-40ec-9326-bef20c29717e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.147592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.148732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.149300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.149769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] PATH_INFO: `/v3/projects/3afa691f89a640a688795a3c1005abb4` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.151847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afa691f89a640a688795a3c1005abb4)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.155621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.180710 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09369e28336d4f49a60ce03c3a3009ac', 'request_id': 'req-fd0d894c-0091-4861-b941-57117a89a315'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.190932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09369e28336d4f49a60ce03c3a3009ac', 'request_id': 'req-fd0d894c-0091-4861-b941-57117a89a315'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.204943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09369e28336d4f49a60ce03c3a3009ac', 'request_id': 'req-fd0d894c-0091-4861-b941-57117a89a315'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.224801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] The token cache is being invalidate because project 09369e28336d4f49a60ce03c3a3009ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:18.225734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.227060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd0d894c-0091-4861-b941-57117a89a315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.230226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1259/2519] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:17 2022] DELETE /identity/v3/projects/09369e28336d4f49a60ce03c3a3009ac => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:18.237849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27997183-cee5-44d5-8a5a-1e647be53778 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.238292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27997183-cee5-44d5-8a5a-1e647be53778 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.238746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27997183-cee5-44d5-8a5a-1e647be53778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.244495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afa691f89a640a688795a3c1005abb4', 'request_id': 'req-ecbda469-3617-4bcc-9228-e342511b030f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.261901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afa691f89a640a688795a3c1005abb4', 'request_id': 'req-ecbda469-3617-4bcc-9228-e342511b030f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.277911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afa691f89a640a688795a3c1005abb4', 'request_id': 'req-ecbda469-3617-4bcc-9228-e342511b030f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.295088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-27997183-cee5-44d5-8a5a-1e647be53778 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:18.298550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-27997183-cee5-44d5-8a5a-1e647be53778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:18.318424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] The token cache is being invalidate because project 3afa691f89a640a688795a3c1005abb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:18.320930 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.323899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ecbda469-3617-4bcc-9228-e342511b030f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:18.326771 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1261/2520] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:18 2022] DELETE /identity/v3/projects/3afa691f89a640a688795a3c1005abb4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:18.335028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862cf4-62ea-45b3-a3e0-f236758ab977 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.335464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862cf4-62ea-45b3-a3e0-f236758ab977 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.336109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c862cf4-62ea-45b3-a3e0-f236758ab977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.392871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9c862cf4-62ea-45b3-a3e0-f236758ab977 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:18.400116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9c862cf4-62ea-45b3-a3e0-f236758ab977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:18.475280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1260/2521] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:18.489564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.489878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.490236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.529359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:18.531697 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:18.551118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1261/2522] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:18.557253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.576666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae83d-b9eb-4c37-b0d2-5cbd5e2742ff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.586427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.587211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.587822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.588410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.590005 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:18.590903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.593514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.624668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1262/2523] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:18.630302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.648518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1262/2524] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:18.649878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0607c2-857c-4590-adc8-28e9ba2ad995 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.656389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.657414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.657711 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.658117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.658796 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.660868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.661084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.661367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.661708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.697254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:18.700409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:18.702801 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:18.717072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1263/2525] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:18.721263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.721759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1263/2526] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:18.727964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.745663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4346e9b-5ae2-4d7e-b4ba-d3b5e73ef5cb None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.763188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:18.856971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b26057-4dcf-46db-b661-2ddb9528e833 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.862063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.862706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.862985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.863259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.864640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.873640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.903682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.904805 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.905397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.905744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.907580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:18.907985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.911742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.930685 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1264/2527] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:18.932873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.951865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ac62f-f898-48f7-965a-ab8abd718ef6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:18.952786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1264/2528] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:41:18.964845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:18.965837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:18.966052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:18.966629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:18.967510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:18.970595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:18.976735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:18.985403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1265/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:18.990110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.001745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cf969-6890-4bd6-94a0-e6940d5cbbc4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.005690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d61d-1a6d-438e-8ee8-ba951509ab7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.011956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.012195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.013034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.013423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.013797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.014417 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.014696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.014875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.015551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.016682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.017528 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.023010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.043041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1266/2530] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:19.048048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.070706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a46f5-8843-4120-9d5f-72e2339aa496 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.085337 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.086502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.086966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.087450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] PATH_INFO: `/v3/projects/d8ee70f927b3424f9ec331327802f41e/users/ccfd745684914e9c87db030451813918/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.120780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ee70f927b3424f9ec331327802f41e, user_id=ccfd745684914e9c87db030451813918, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.129756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d6168-df54-407b-a837-08a742489152 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.140219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1265/2531] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:18 2022] POST /identity/v3/projects => generated 407 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:19.142880 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.200262 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1267/2532] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:19 2022] PUT /identity/v3/projects/d8ee70f927b3424f9ec331327802f41e/users/ccfd745684914e9c87db030451813918/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:19.219207 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.220161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.221239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.280345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] MFA Rules not processed for user `ccfd745684914e9c87db030451813918`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:19.282854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:19.318748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e5d1-ff88-46bf-92c1-9504703f451b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.414848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1268/2533] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:19.430343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.432953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:19.450514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.451349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.451768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.452284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.453397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.455647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.490828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1266/2534] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:19 2022] POST /identity/v3/users => generated 304 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:19.492839 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.510234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f9a1-56d4-4e6d-87cf-c3b44d25b1f5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.514183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:19.520296 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.521328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.521793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.522309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.523303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.526309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.548887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1267/2535] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:19.555559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.575138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df4cc-ece3-4193-8280-94b830e2e16f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.581824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.582712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.583079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.583402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.584099 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.586222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.592756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:19.602676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fd28-1c88-4f10-9208-d6a6089b37d9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.611815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1268/2536] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:19.613490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.615298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.616489 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.617232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.618478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.632648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a288f-7bef-4c5d-be47-2eb759679589 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:19.635365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.638997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7218b-97da-4dc1-9807-e34faa61f2f5 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.640399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:19.642107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.642430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.642728 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] PATH_INFO: `/v3/projects/37bf5e9011ee49d986b9d8035003de29/users/cac8bff59aed490fa3a0de3257fce717/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.657069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1269/2537] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:19.670258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bf5e9011ee49d986b9d8035003de29, user_id=cac8bff59aed490fa3a0de3257fce717, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:19.675562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:19.702226 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1269/2538] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:19 2022] PUT /identity/v3/projects/37bf5e9011ee49d986b9d8035003de29/users/cac8bff59aed490fa3a0de3257fce717/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:19.719313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:19.719704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:19.720116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:19.774729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] MFA Rules not processed for user `cac8bff59aed490fa3a0de3257fce717`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:19.776336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:19.960075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:19.974159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1270/2539] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:20.110191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e3a78c-4675-4957-b183-cce57ec56c72 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.118179 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.118815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.119094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.119372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.120091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.122213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.170894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1270/2540] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:19 2022] POST /identity/v3/projects => generated 407 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:20.176788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.197842 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3341cf-58e4-4b52-bad2-ff69a4315bc1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.362715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.363864 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.364343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.364794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.365891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.369631 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.405466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1271/2541] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:20 2022] POST /identity/v3/users => generated 310 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:20.406884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.423533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2067a57-7ad6-4ace-9d46-c97f79a2e58a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.432910 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.433637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.434096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.434690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.435356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.437816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.451642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1271/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:20.459271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.480615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6247b-acac-49ee-9cbf-e873b8425410 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.489395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.490673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.491178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.491660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] PATH_INFO: `/v3/projects/9736709c5c444edfabd52822fbce05b6/users/9ba4ee319b7e48d2a38ea014261ebc70/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.521705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9736709c5c444edfabd52822fbce05b6, user_id=9ba4ee319b7e48d2a38ea014261ebc70, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.526848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.547486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1272/2543] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:20 2022] PUT /identity/v3/projects/9736709c5c444edfabd52822fbce05b6/users/9ba4ee319b7e48d2a38ea014261ebc70/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:20.551945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.565063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa18178-3f33-4e8b-8e1f-ce1032967c9a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.653241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.654262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.654986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.655459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.656776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.659763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3777cb2-2a32-4fec-9b78-5fe895f3b3aa admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.675093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1272/2544] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:20.681298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.693185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32badac-54f9-4f27-8970-2f8ed32e30a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.698203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.698876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.699153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.699427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] PATH_INFO: `/v3/projects/9736709c5c444edfabd52822fbce05b6/users/9ba4ee319b7e48d2a38ea014261ebc70/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.701568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9736709c5c444edfabd52822fbce05b6, user_id=9ba4ee319b7e48d2a38ea014261ebc70, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.704188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:20.711781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:20.718401 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1273/2545] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:20 2022] PUT /identity/v3/projects/9736709c5c444edfabd52822fbce05b6/users/9ba4ee319b7e48d2a38ea014261ebc70/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:20.731262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.731589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.731817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.772007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] MFA Rules not processed for user `9ba4ee319b7e48d2a38ea014261ebc70`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:20.774422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:20.833280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:20.878002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1273/2546] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:20 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:20.895723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:20.898680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:20.965690 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f6b8f-c1ef-45aa-85c0-7ebb8f7dedb6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:20.972816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:20.974622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:20.974622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:20.974918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:20.989838 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:20.991383 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:21.002963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1274/2547] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:20 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:21.199486 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:21.224860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b4e1-5456-42a5-b8d6-9f83d59c8a34 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:21.229727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:21.230384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:21.230660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:21.230942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:21.322970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:21.326341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0a2e1-08f6-4043-a854-89fbaf176366 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:21.346163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1274/2548] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:21 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:23.915748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:23.999812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc024e-b1d9-41a1-8188-801b7846467a service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:24.011877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:24.012539 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:24.012857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:24.013150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] PATH_INFO: `/v3/users/ccfd745684914e9c87db030451813918` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:24.014367 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccfd745684914e9c87db030451813918)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:24.016338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:24.080954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccfd745684914e9c87db030451813918'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.091609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccfd745684914e9c87db030451813918'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.103275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccfd745684914e9c87db030451813918'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.104060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccfd745684914e9c87db030451813918'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.126505 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccfd745684914e9c87db030451813918'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.133728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1275/2549] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:23 2022] DELETE /identity/v3/users/ccfd745684914e9c87db030451813918 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:24.138782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:24.139446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:24.139446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:24.160431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:24.161660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:24.230159 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1275/2550] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:24.243769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:24.246180 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:24.296172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:24.358911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06d37d-c186-4bdf-a8db-f285663543f7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:24.367827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:24.368753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:24.369486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:24.369962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:24.388211 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:24.390675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f350c-7fc6-4422-9ede-709c1d5d7f38 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:24.415512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1276/2551] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:24.785685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:24.788146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:24.808308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694859-18d1-494f-87c0-89d5b62bbf21 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:24.814782 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:24.815840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:24.816115 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:24.816467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] PATH_INFO: `/v3/projects/d8ee70f927b3424f9ec331327802f41e` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:24.817824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ee70f927b3424f9ec331327802f41e)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:24.820073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:24.880809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ee70f927b3424f9ec331327802f41e', 'request_id': 'req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.895340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ee70f927b3424f9ec331327802f41e', 'request_id': 'req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.906227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ee70f927b3424f9ec331327802f41e', 'request_id': 'req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.908004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:24.924858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] The token cache is being invalidate because project d8ee70f927b3424f9ec331327802f41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:24.925146 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.925812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3ac66449-4ca5-4b40-bfe1-6cc134885bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:24.927356 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1276/2552] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:24 2022] DELETE /identity/v3/projects/d8ee70f927b3424f9ec331327802f41e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:24.934432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:24.934981 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:24.935187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:24.956723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:24.958554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:25.049353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1277/2553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:25.057434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.057719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.057993 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.088081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:25.089442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:25.098727 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1277/2554] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:25.103313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.115529 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7f0ec-7a54-47c2-97eb-761f8eb117e6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.122653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.123353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.123716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.124203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.125321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:25.125779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.128243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.148974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1278/2555] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:41:25.153312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.163821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9a7a9-1aee-4c3e-bc9c-512e5149357f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.168868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.169482 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.169756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.170030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.170678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.172652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.211180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1278/2556] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:25.215730 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.226867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b33783-50ce-4402-bd1b-8d7b8d8a1362 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.298819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.299462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.299803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.300104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.300895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.302901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.340484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1279/2557] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:25.342515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.358806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d978c-c9b0-40e4-958b-99cc7e1b8bb3 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.364842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.365455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.365733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.366003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.366655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.368293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.379706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1279/2558] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:25.384473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.401152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6797745-7a1b-4418-950f-7db717d541c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.408794 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.409513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.409940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.410902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.412133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.415414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d05ba-f1e4-4e94-8428-cfc9679a8207 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.435079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1280/2559] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:25.441195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.455532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46990-8958-457a-99f7-4c9c2360747a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.463954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.465458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.465925 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.466375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] PATH_INFO: `/v3/projects/c2998472f8124b359f8c4cc1e7cc7842/users/25176994216643c684ab2090dc764e01/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.490914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2998472f8124b359f8c4cc1e7cc7842, user_id=25176994216643c684ab2090dc764e01, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.494393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.509660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1280/2560] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:25 2022] PUT /identity/v3/projects/c2998472f8124b359f8c4cc1e7cc7842/users/25176994216643c684ab2090dc764e01/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:25.524803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.525046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.525454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.560769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] MFA Rules not processed for user `25176994216643c684ab2090dc764e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:25.562590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:25.596852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:25.667319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1281/2561] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:25.672134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.788077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5b4df-9ca1-4280-94db-b343e58559d2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.796454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.797677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.797976 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.798455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.799687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.803990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.849319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1281/2562] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:25.854793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:25.870770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5c008-2a47-4a5b-87f7-1b1c0df95c4c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:25.953819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:25.954761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:25.955249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:25.955736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:25.956904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:25.960619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:25.997457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1282/2563] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:25 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:26.002443 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:26.023495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6e6c-10df-4c33-81cc-813806e35ba6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:26.029844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:26.030547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.030860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.031223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.031919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:26.033666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:26.047750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1282/2564] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:26.052467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:26.064164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328cf48-76d8-41be-a30f-958348821d62 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:26.069903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:26.071294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.071588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.072004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] PATH_INFO: `/v3/projects/c89bf5e4362d4658b83d1857b7e706be/users/b7f5f792c5e945548101653901af1d80/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.093303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89bf5e4362d4658b83d1857b7e706be, user_id=b7f5f792c5e945548101653901af1d80, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:26.099042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:26.137150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1283/2565] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:26 2022] PUT /identity/v3/projects/c89bf5e4362d4658b83d1857b7e706be/users/b7f5f792c5e945548101653901af1d80/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:26.142672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:26.159035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13cec7-f95a-4a7c-82a7-2c279e8ef785 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:26.253955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:26.255760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.256248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.256723 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.257885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:26.260924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe3467-b842-4a53-b214-0117c982efe2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:26.269850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:26.277978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1283/2566] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:26.285578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:26.307857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b03a7-b4eb-476d-9ecc-d7a166ccb53d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:26.318582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:26.319736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.320224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.320781 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] PATH_INFO: `/v3/projects/c89bf5e4362d4658b83d1857b7e706be/users/b7f5f792c5e945548101653901af1d80/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.324683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89bf5e4362d4658b83d1857b7e706be, user_id=b7f5f792c5e945548101653901af1d80, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:26.327051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:26.352647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1284/2567] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:26 2022] PUT /identity/v3/projects/c89bf5e4362d4658b83d1857b7e706be/users/b7f5f792c5e945548101653901af1d80/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:26.365490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.366015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.366526 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.432141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] MFA Rules not processed for user `b7f5f792c5e945548101653901af1d80`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:26.432141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:26.602934 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1284/2568] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:26.626319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:26.629120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:26.786490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c02688-3f75-4dd6-8e1d-94109e7b2d19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:26.801629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:26.803319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:26.803979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:26.804803 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:26.829751 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:26.833841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:26.869604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1285/2569] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:27.106624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:27.108259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:27.133995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64906f5-ede5-40e5-b634-d0ca243855d4 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:27.139791 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:27.140804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:27.141262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:27.141553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:27.265711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:27.267434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb22a23-88d5-4fc1-b10b-79c20d33a2fb service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:27.290144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1285/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:28.986984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:28.988549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:29.017813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:29.092949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f6e9f-924e-4fd9-9277-416e701951a8 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:29.105635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:29.106852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:29.107441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:29.108082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] PATH_INFO: `/v3/users/cac8bff59aed490fa3a0de3257fce717` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:29.110918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac8bff59aed490fa3a0de3257fce717)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:29.115010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:29.196347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac8bff59aed490fa3a0de3257fce717'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:29.222490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac8bff59aed490fa3a0de3257fce717'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:29.237859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac8bff59aed490fa3a0de3257fce717'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:29.241110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac8bff59aed490fa3a0de3257fce717'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:29.269065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac8bff59aed490fa3a0de3257fce717'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:29.282464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1286/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:28 2022] DELETE /identity/v3/users/cac8bff59aed490fa3a0de3257fce717 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:29.288894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:29.289722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:29.289722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:29.315967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:29.318258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:29.401216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1286/2572] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:29.419507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:29.421062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:29.494588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69c03-7ef3-4943-add6-f136ebe9c5ac admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:29.498653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:29.499274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:29.499549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:29.499852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:29.512126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:29.513847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:29.524898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1287/2573] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:29.970494 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:29.972183 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:29.989756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283eec7-6de2-427e-9ae1-5f02b0192de8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:29.997295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:29.998127 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:29.998636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:29.998891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] PATH_INFO: `/v3/projects/37bf5e9011ee49d986b9d8035003de29` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:30.000702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bf5e9011ee49d986b9d8035003de29)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:30.003999 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:30.047076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:30.110426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bf5e9011ee49d986b9d8035003de29', 'request_id': 'req-b23ed941-d097-4026-b8b7-6d827be753de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.126579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bf5e9011ee49d986b9d8035003de29', 'request_id': 'req-b23ed941-d097-4026-b8b7-6d827be753de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.134135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bf5e9011ee49d986b9d8035003de29', 'request_id': 'req-b23ed941-d097-4026-b8b7-6d827be753de'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.151651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] The token cache is being invalidate because project 37bf5e9011ee49d986b9d8035003de29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:30.151904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.152578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.154184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1287/2574] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:29 2022] DELETE /identity/v3/projects/37bf5e9011ee49d986b9d8035003de29 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:30.159583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:30.177889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3afae8-2c68-485e-b002-cdf69a4dcebe service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:30.233958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:30.268214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:30.268877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:30.269153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:30.269422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] PATH_INFO: `/v3/users/9ba4ee319b7e48d2a38ea014261ebc70` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:30.271045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba4ee319b7e48d2a38ea014261ebc70)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:30.273014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:30.326419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba4ee319b7e48d2a38ea014261ebc70'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.334384 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba4ee319b7e48d2a38ea014261ebc70'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.342597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba4ee319b7e48d2a38ea014261ebc70'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.343294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba4ee319b7e48d2a38ea014261ebc70'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.358122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba4ee319b7e48d2a38ea014261ebc70'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.369754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1288/2575] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:30 2022] DELETE /identity/v3/users/9ba4ee319b7e48d2a38ea014261ebc70 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:30.568739 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:30.572841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:30.590589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ed941-d097-4026-b8b7-6d827be753de admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:30.709891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:30.711734 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:30.712333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:30.713063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] PATH_INFO: `/v3/users/25176994216643c684ab2090dc764e01` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:30.716957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25176994216643c684ab2090dc764e01)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:30.720132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:30.816058 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25176994216643c684ab2090dc764e01'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.829419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:30.830968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:30.843178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25176994216643c684ab2090dc764e01'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.848402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbdec1-afbd-4b77-b242-88a5ee14cb12 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:30.880271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:30.882093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:30.883139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:30.883700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] PATH_INFO: `/v3/projects/9736709c5c444edfabd52822fbce05b6` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:30.887947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9736709c5c444edfabd52822fbce05b6)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:30.892094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:30.893604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25176994216643c684ab2090dc764e01'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.897586 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25176994216643c684ab2090dc764e01'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.898932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-55bf1a5a-3362-4182-af74-40f13aeabf79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25176994216643c684ab2090dc764e01'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:30.926013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1288/2576] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:30 2022] DELETE /identity/v3/users/25176994216643c684ab2090dc764e01 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:30.932400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:30.933249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:30.933692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:30.965063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:30.970527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.007296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9736709c5c444edfabd52822fbce05b6', 'request_id': 'req-e550d5b7-d176-475d-a07b-55b1dbc7c13c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.018634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9736709c5c444edfabd52822fbce05b6', 'request_id': 'req-e550d5b7-d176-475d-a07b-55b1dbc7c13c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.024396 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:31.039281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9736709c5c444edfabd52822fbce05b6', 'request_id': 'req-e550d5b7-d176-475d-a07b-55b1dbc7c13c'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.059871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] The token cache is being invalidate because project 9736709c5c444edfabd52822fbce05b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:31.060261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.061726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-e550d5b7-d176-475d-a07b-55b1dbc7c13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.066212 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1289/2577] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:30 2022] DELETE /identity/v3/projects/9736709c5c444edfabd52822fbce05b6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:31.071934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703df2ad-100d-430f-a9ac-08fcc85c6db1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.072504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703df2ad-100d-430f-a9ac-08fcc85c6db1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.073252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703df2ad-100d-430f-a9ac-08fcc85c6db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.090290 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1289/2578] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:31.102339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-703df2ad-100d-430f-a9ac-08fcc85c6db1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:31.103827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-703df2ad-100d-430f-a9ac-08fcc85c6db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.104225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.105832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.154420 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:31.203443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1290/2579] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:31.216472 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.216958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.217409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.230909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c609f-b2fe-4664-acd6-c3a18a6ad0b1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.245170 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:31.247246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.259488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1291/2580] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:31.265285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.277999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52aae6-5b45-4fa0-84b8-2b3258ff12cd None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.283477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.284126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.284520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.284725 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.286087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:31.286469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.288302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.307841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1292/2581] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:41:31.309883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.311250 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.311883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.312438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.314259 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.330740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.332306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.333975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a2627-db26-4ad8-ad2d-bf90ef364a73 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.342852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.343921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.344498 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.345184 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.346556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.348241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1290/2582] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:31.351223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.524026 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1293/2583] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:31 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:31.531034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.543543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8844606-236c-4a5d-af13-184d7fa738a3 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.635350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.636565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.636565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.636943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.637757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.639811 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.669944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1291/2584] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:31 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:31.674001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.689279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bda7a8-18ea-4e94-8aa0-168f81471de9 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.700993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.701769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.702163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.702642 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.703614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.705824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.721400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1294/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:31.728540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.744615 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaadfe-b420-4650-8039-b9311f553c87 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.751815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.752733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.753157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.753554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.754568 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.756850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.776631 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1292/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:31.781404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.800851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a77f5-a615-49b8-9e03-7e5924d8be6b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.804982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:31.808014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.821792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.823244 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.823813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.824307 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] PATH_INFO: `/v3/projects/9bff2dbd28314ed5ac9852e9238eee3a/users/e70207f0ac9845098851dfb248c7e91f/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.827516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:31.833843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786d857-5ee7-4526-b0ef-ece80fa4ea75 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:31.846195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:31.847197 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.847857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.848540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] PATH_INFO: `/v3/projects/c2998472f8124b359f8c4cc1e7cc7842` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.851027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2998472f8124b359f8c4cc1e7cc7842)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.851334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bff2dbd28314ed5ac9852e9238eee3a, user_id=e70207f0ac9845098851dfb248c7e91f, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:31.859863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.861720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b348d-7cc3-4d27-88bc-2fe21af766a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:31.885303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1295/2587] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:31 2022] PUT /identity/v3/projects/9bff2dbd28314ed5ac9852e9238eee3a/users/e70207f0ac9845098851dfb248c7e91f/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:31.900050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:31.900338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:31.900758 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:31.950072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] MFA Rules not processed for user `e70207f0ac9845098851dfb248c7e91f`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:31.951330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:31.967128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2998472f8124b359f8c4cc1e7cc7842', 'request_id': 'req-11fb21a7-99aa-4ea5-945d-cf03baceca83'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.978849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2998472f8124b359f8c4cc1e7cc7842', 'request_id': 'req-11fb21a7-99aa-4ea5-945d-cf03baceca83'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:31.988311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2998472f8124b359f8c4cc1e7cc7842', 'request_id': 'req-11fb21a7-99aa-4ea5-945d-cf03baceca83'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.010691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] The token cache is being invalidate because project c2998472f8124b359f8c4cc1e7cc7842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:32.011044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.012018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.013975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1293/2588] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:31 2022] DELETE /identity/v3/projects/c2998472f8124b359f8c4cc1e7cc7842 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:32.018593 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.021017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:32.080848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1296/2589] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:32.086943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.089583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:32.147017 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb21a7-99aa-4ea5-945d-cf03baceca83 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.153817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.154575 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.154858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.155156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] PATH_INFO: `/v3/users/b7f5f792c5e945548101653901af1d80` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.156363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f5f792c5e945548101653901af1d80)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.158349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.187282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:32.221389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ea5d5-20f2-4dd3-855d-2396c315b6c7 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.228510 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.229171 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.229460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.229733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.230773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.232422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.251284 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f5f792c5e945548101653901af1d80'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.276519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f5f792c5e945548101653901af1d80'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.290087 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1297/2590] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:32 2022] POST /identity/v3/projects => generated 365 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:32.291441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.297621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f5f792c5e945548101653901af1d80'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.298764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f5f792c5e945548101653901af1d80'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.299606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f5f792c5e945548101653901af1d80'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:32.320838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1294/2591] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:32 2022] DELETE /identity/v3/users/b7f5f792c5e945548101653901af1d80 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:32.407901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584202f-a30c-4029-983f-1b3bb4b6431f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.503220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.504895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.505859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.505859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.507048 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.510793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.549509 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1298/2592] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:32 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:32.562523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.565508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:32.584551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf815-58cb-4eed-8608-698fa961d176 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.593178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.594431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.594994 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.595434 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.596626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.599217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.601153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:32.619172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1295/2593] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:32.620456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.642154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f141-40cc-40e7-ae65-8a6797064089 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.650437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.651562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.652059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.652528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] PATH_INFO: `/v3/projects/1c872137d05f46a1a3676de7b5a01b91/users/0067d8b66756417a92e50ab88251b0ec/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.679344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c872137d05f46a1a3676de7b5a01b91, user_id=0067d8b66756417a92e50ab88251b0ec, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.682679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.704327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1299/2594] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:32 2022] PUT /identity/v3/projects/1c872137d05f46a1a3676de7b5a01b91/users/0067d8b66756417a92e50ab88251b0ec/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:32.708914 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.731998 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c61f1-220e-40c6-bbf1-e237789ac50e admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.819141 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.820002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.820186 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.820515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.821990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.827674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba5e65-a980-4b38-9572-b16b36729de8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.846269 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1296/2595] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:32.851997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:32.870588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3de485-723e-40b0-97ab-7607ce82a9a3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:32.877063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:32.878179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.878601 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.879104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] PATH_INFO: `/v3/projects/1c872137d05f46a1a3676de7b5a01b91/users/0067d8b66756417a92e50ab88251b0ec/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.882965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c872137d05f46a1a3676de7b5a01b91, user_id=0067d8b66756417a92e50ab88251b0ec, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:32.886257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:32.905770 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1300/2596] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:32 2022] PUT /identity/v3/projects/1c872137d05f46a1a3676de7b5a01b91/users/0067d8b66756417a92e50ab88251b0ec/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:32.923601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:32.923966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:32.924351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:32.975935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] MFA Rules not processed for user `0067d8b66756417a92e50ab88251b0ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:32.977260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.025786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.027732 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.098666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:33.131195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1297/2597] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:33.145369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4b830-0d18-4a55-a654-0135dee7fe85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.149609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.151436 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.152972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.153889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.154277 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.154639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] PATH_INFO: `/v3/projects/c89bf5e4362d4658b83d1857b7e706be` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.156251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89bf5e4362d4658b83d1857b7e706be)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:33.159164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:33.249260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db9ae0-4664-4775-9016-abbdd84332ae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.251582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89bf5e4362d4658b83d1857b7e706be', 'request_id': 'req-fd590593-b53e-446e-8391-6072824b20dc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:33.262571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89bf5e4362d4658b83d1857b7e706be', 'request_id': 'req-fd590593-b53e-446e-8391-6072824b20dc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:33.282668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89bf5e4362d4658b83d1857b7e706be', 'request_id': 'req-fd590593-b53e-446e-8391-6072824b20dc'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:33.310133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] The token cache is being invalidate because project c89bf5e4362d4658b83d1857b7e706be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:33.311334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:33.314319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fd590593-b53e-446e-8391-6072824b20dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:33.317715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1301/2598] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:33 2022] DELETE /identity/v3/projects/c89bf5e4362d4658b83d1857b7e706be => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:33.325106 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23306d2b-935d-410a-8fc4-b5c686d49f27 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.326217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23306d2b-935d-410a-8fc4-b5c686d49f27 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.327096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23306d2b-935d-410a-8fc4-b5c686d49f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.351643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:33.385132 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-23306d2b-935d-410a-8fc4-b5c686d49f27 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:33.388042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-23306d2b-935d-410a-8fc4-b5c686d49f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.398704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.399835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.400134 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.400409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.402564 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.558287 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1302/2599] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:33.561270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.561646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.561990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.599596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:33.602135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.609881 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:33.613110 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:33.618403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1303/2600] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:33.624645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.632681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1298/2601] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:33.638121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac2d80-6dc5-4ee8-a1b0-bd68c71adf06 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.645696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.646491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.646916 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.647300 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.648068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:33.648465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:33.651731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:33.672164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1304/2602] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:33.678328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.718731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22900887-67ea-497d-bfec-c7fe2c2f4dbb service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.736040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.739102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.740355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.741817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.743025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:33.745733 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:33.779130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.781518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.800431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1299/2603] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:33 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:33.802135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7de552-4cb0-451f-af27-808766730aee admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.803787 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:33.880548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:33.890942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.891915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.892282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.892680 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.895142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:33.971690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9687b5a-2f5d-4de6-8e3b-3cf83784f8f0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:33.988764 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:33.991455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:33.992279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:33.994577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:33.996060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.002314 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.044362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.047185 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.059616 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1300/2604] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:33 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:34.076755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.077198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1305/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:34.106267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ef2b4-ae72-4f97-91d9-09c863a11423 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.115841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.116515 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.117029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.117400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.118516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.122249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.139795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1301/2606] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:34.145121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.163518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5447744-b71b-4071-a2ee-c6088f3209ec service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.170832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.171836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.172328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.172852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.174178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.177706 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d0771-6ff8-4b1d-8510-53c50a70dfa5 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.200747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1306/2607] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:34.202694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.215367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bb6bb-c4ab-42e4-871b-de50d4d1ba2d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.221248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.222107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.223086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.223609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] PATH_INFO: `/v3/projects/74a68ff93b9049ae87811189cb4aa250/users/877d7bc01f4f4145a8a6cd9348000425/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.247457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a68ff93b9049ae87811189cb4aa250, user_id=877d7bc01f4f4145a8a6cd9348000425, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.252341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.276380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1302/2608] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:34 2022] PUT /identity/v3/projects/74a68ff93b9049ae87811189cb4aa250/users/877d7bc01f4f4145a8a6cd9348000425/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:34.296643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.298150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.298593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.348728 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] MFA Rules not processed for user `877d7bc01f4f4145a8a6cd9348000425`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:34.350883 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:34.481366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1307/2609] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:34.501936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.530240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:34.599984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058675c-d49a-4d5f-b117-ea4f17dd8d6c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.606699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.607332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.607614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.608092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.608795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.610759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.648461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1303/2610] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:34 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:34.654029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.674050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2676c41-5576-4644-acc2-9c78ad6ce823 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.757572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.758206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.758484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.758779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.759477 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.762192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.789714 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1308/2611] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:34 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:34.794859 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.810501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009af84a-c1ec-416e-a384-25dedfb870f6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.819162 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.820243 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.820732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.821126 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.821955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.823753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.839605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1304/2612] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:34.845782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.862525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306eff-e69e-442e-98d0-e992ae6aaa19 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:34.868812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:34.869582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:34.870238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:34.870443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] PATH_INFO: `/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f/users/1faff6253e114bb38adc2ea5a9206317/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:34.883626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:34.896930 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02bef536dea4328a3fa6f4f2eebf78f, user_id=1faff6253e114bb38adc2ea5a9206317, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:34.902128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:34.930608 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1309/2613] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:34 2022] PUT /identity/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f/users/1faff6253e114bb38adc2ea5a9206317/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:34.935899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:34.952066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb09825-6208-45c4-88fc-2a33f2b3a10a admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:35.042216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:35.043362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.043754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.044218 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.045487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:35.048323 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d22e-8c58-4807-b5aa-637cb552cf0b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:35.068360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1305/2614] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:35.074423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:35.090644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159914ef-0eae-4d9a-ba42-b30e227c5a4b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:35.100260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:35.100260 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.100923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.101066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] PATH_INFO: `/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f/users/1faff6253e114bb38adc2ea5a9206317/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.104003 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02bef536dea4328a3fa6f4f2eebf78f, user_id=1faff6253e114bb38adc2ea5a9206317, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:35.108704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:35.129828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1310/2615] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:35 2022] PUT /identity/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f/users/1faff6253e114bb38adc2ea5a9206317/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:35.146848 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c067c2-9867-43ae-a692-70d9eb3c6e94 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.147310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c067c2-9867-43ae-a692-70d9eb3c6e94 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.147642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c067c2-9867-43ae-a692-70d9eb3c6e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.208196 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-61c067c2-9867-43ae-a692-70d9eb3c6e94 None None] MFA Rules not processed for user `1faff6253e114bb38adc2ea5a9206317`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:35.209832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-61c067c2-9867-43ae-a692-70d9eb3c6e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:35.353599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1306/2616] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:35.374294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:35.473464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.473929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.474380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.489116 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090cced-681f-4549-a13b-649ce88b24b2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:35.495084 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:35.496053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.496720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.497009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.513109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:35.514046 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:35.515653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:35.516405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:35.529030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:35.532956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1311/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:35.649969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1307/2618] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:41:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:35.667660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:35.674849 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:35.755554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:35.874971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749cf77-2224-4c90-9358-b85edbbb7e3d service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:35.886846 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:35.888354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.888907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.889415 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.914887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:35.917942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9d0e-8aa5-420a-95a0-713a0202ddaa service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:35.927477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fb96c-f4c6-40c7-9ca8-77dce12ab360 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:35.933855 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1312/2619] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:41:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:41:35.941711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:35.943263 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:35.943783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:35.944103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] PATH_INFO: `/v3/users/877d7bc01f4f4145a8a6cd9348000425` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:35.945573 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877d7bc01f4f4145a8a6cd9348000425)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:35.949090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:36.028652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877d7bc01f4f4145a8a6cd9348000425'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:36.061884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877d7bc01f4f4145a8a6cd9348000425'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:36.076815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877d7bc01f4f4145a8a6cd9348000425'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:36.078219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877d7bc01f4f4145a8a6cd9348000425'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:36.079681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877d7bc01f4f4145a8a6cd9348000425'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:36.112864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1308/2620] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:35 2022] DELETE /identity/v3/users/877d7bc01f4f4145a8a6cd9348000425 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:36.118172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ae853-6d7f-4e8e-99f5-0d30cabfb79d None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:36.118448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ae853-6d7f-4e8e-99f5-0d30cabfb79d None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:36.118798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ae853-6d7f-4e8e-99f5-0d30cabfb79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:36.167042 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-0b1ae853-6d7f-4e8e-99f5-0d30cabfb79d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:36.168898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0b1ae853-6d7f-4e8e-99f5-0d30cabfb79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:36.304645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1313/2621] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:36.319164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:36.321495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:36.396454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:36.433803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53517d1-476c-411a-af6c-3ed73bc5a963 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:36.440540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:36.442081 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:36.442567 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:36.443048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:36.460749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:36.463478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:36.479344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1309/2622] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:36.841481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:36.842258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:36.842449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:36.866715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:36.876547 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:36.879227 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:36.937746 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:36.940729 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:36.967071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda774c-65d5-4606-94dd-f377905fce05 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:36.986854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:36.987864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:36.988891 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:36.989563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] PATH_INFO: `/v3/projects/74a68ff93b9049ae87811189cb4aa250` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:36.991666 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a68ff93b9049ae87811189cb4aa250)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:36.994874 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:37.003053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1314/2623] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:41:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:37.026268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:37.028907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:37.088467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a68ff93b9049ae87811189cb4aa250', 'request_id': 'req-4578accb-524a-4119-bc94-5849e50f68b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.103532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a68ff93b9049ae87811189cb4aa250', 'request_id': 'req-4578accb-524a-4119-bc94-5849e50f68b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.121978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a68ff93b9049ae87811189cb4aa250', 'request_id': 'req-4578accb-524a-4119-bc94-5849e50f68b9'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.137579 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] The token cache is being invalidate because project 74a68ff93b9049ae87811189cb4aa250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:37.138031 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.138974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.140887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1310/2624] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:36 2022] DELETE /identity/v3/projects/74a68ff93b9049ae87811189cb4aa250 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:37.143998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c0662-2166-4089-8d57-9d1743b9668c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:37.145080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:37.146637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:37.239379 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578accb-524a-4119-bc94-5849e50f68b9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:37.243110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:37.244862 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:37.245629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:37.245915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:37.246274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:37.246558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:37.246836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] PATH_INFO: `/v3/users/1faff6253e114bb38adc2ea5a9206317` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:37.247160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:37.248003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1faff6253e114bb38adc2ea5a9206317)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:37.250027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:37.251254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:37.348223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1faff6253e114bb38adc2ea5a9206317'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.361338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:37.365473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:37.367139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d612fd-21b6-4f67-b6c4-ddbbe94f135b service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:37.371810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:37.373410 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1faff6253e114bb38adc2ea5a9206317'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.384000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1315/2625] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:41:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:41:37.387273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1faff6253e114bb38adc2ea5a9206317'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.388508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1faff6253e114bb38adc2ea5a9206317'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.389540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1faff6253e114bb38adc2ea5a9206317'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:37.414909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1311/2626] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:37 2022] DELETE /identity/v3/users/1faff6253e114bb38adc2ea5a9206317 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:37.788967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997e31-ebdc-4388-b826-d20557981fff None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:37.789406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997e31-ebdc-4388-b826-d20557981fff None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:37.789830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997e31-ebdc-4388-b826-d20557981fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:37.822606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-81997e31-ebdc-4388-b826-d20557981fff None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:37.824662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81997e31-ebdc-4388-b826-d20557981fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:37.900059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:37.902255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:37.974749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1316/2627] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:41:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:37.993114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997e31-ebdc-4388-b826-d20557981fff None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:37.994357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b69d41-164a-4fa9-83a9-08b9595f2270 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:37.995580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-81997e31-ebdc-4388-b826-d20557981fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.005962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.007062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.009613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.009613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] PATH_INFO: `/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.011032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02bef536dea4328a3fa6f4f2eebf78f)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.013684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.112566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02bef536dea4328a3fa6f4f2eebf78f', 'request_id': 'req-637bc065-80c3-43c7-82b9-c76b271597cc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:38.136047 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02bef536dea4328a3fa6f4f2eebf78f', 'request_id': 'req-637bc065-80c3-43c7-82b9-c76b271597cc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:38.139532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997e31-ebdc-4388-b826-d20557981fff None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.149330 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02bef536dea4328a3fa6f4f2eebf78f', 'request_id': 'req-637bc065-80c3-43c7-82b9-c76b271597cc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:38.172627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] The token cache is being invalidate because project e02bef536dea4328a3fa6f4f2eebf78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:38.173451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:38.176442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-637bc065-80c3-43c7-82b9-c76b271597cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:38.190178 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1312/2628] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:37 2022] DELETE /identity/v3/projects/e02bef536dea4328a3fa6f4f2eebf78f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:38.197800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.198369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.198831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.233596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:38.235921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.286119 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.287054 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.287746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.288530 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.291717 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.363883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1313/2629] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:38.365132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.369117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.400005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:38.404249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:38.426926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.428791 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eec7d7-9ff5-474a-acf5-7eeb0a362fff service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.442891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1317/2630] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:41:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:41:38.446465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.446465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.446465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.490497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:38.492342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.510857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1318/2631] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:38.518687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.523525 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ece4ae-0ffa-46e4-9fa1-f519d3010ffd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.531234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.532415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.533149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.533633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.538102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:38.541878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7ef5-d9be-4815-aa83-fb437e1c4943 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.552528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.555314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.556471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.556471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.558909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:38.559723 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.563719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.592302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1319/2632] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:41:38.599230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.617928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee146-e375-459f-9cfc-4a9a0af4de50 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.629270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.630929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.631354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.631868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.632800 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.638137 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.681965 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.683834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.708201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1320/2633] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:38 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:38.708503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1314/2634] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:38.714853 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.824011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c674bba-f5d0-4763-8568-48daf37715a8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.830944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.831328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.831663 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.832556 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.833423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.836724 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.862478 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1321/2635] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:38 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:38.867795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.892021 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dbadb-b75b-4143-a01a-187dd87c6947 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.901276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.902424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.902926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.903423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.904660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.907843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.929663 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1315/2636] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:38.936058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:38.952389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45dab9-f07c-4647-afd3-d7c84f7a0d27 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:38.961389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:38.962161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:38.962495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:38.962828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:38.963752 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:38.965958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e295fc-e71f-471a-9b15-2914c878d255 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:38.982727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1322/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:38.987008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.001106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5935a-784e-45ec-a195-1d495b87af91 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.007365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.008431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.008759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.009155 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] PATH_INFO: `/v3/projects/0a9ebc9dbe8145639975b429baa99cc5/users/9aa4240e77854e8a9fec966217a5bda0/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.031785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9ebc9dbe8145639975b429baa99cc5, user_id=9aa4240e77854e8a9fec966217a5bda0, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.036400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270a9c8-d098-454a-8588-e0b6b172a993 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.063511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1316/2638] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:38 2022] PUT /identity/v3/projects/0a9ebc9dbe8145639975b429baa99cc5/users/9aa4240e77854e8a9fec966217a5bda0/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:39.077389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.078881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.079627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.115655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.116107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.116804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.154365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] MFA Rules not processed for user `9aa4240e77854e8a9fec966217a5bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:39.156481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:39.162252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:39.165212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:39.289486 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1323/2639] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:39.299400 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.305374 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1317/2640] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:41:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:39.335339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:39.335699 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.341274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:39.433207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:39.451630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68356d-dc45-4918-a572-495fb9627cf3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.466168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.467196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.467552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.468521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.469270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.471520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.479856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d6665-1fee-4a0b-999d-632b5fc2754d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.491297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.492414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.492861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.493345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.509987 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.513870 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.526162 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1324/2641] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:39 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:39.527892 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.529077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1318/2642] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:41:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:41:39.634681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5946c6-8ada-4e63-9fa1-00435f19ba11 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.641243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.641867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.642310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.642600 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.643246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.645320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.675061 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1325/2643] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:39 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:39.683209 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.701650 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a35553-7487-4e97-af40-4536e235d45b service swift] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.708984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.709776 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.710283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.710714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.711753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.714166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.732518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1319/2644] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:39.741793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.756932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628de2a-2e28-4dc9-87d1-7d81d72ef770 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.765375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.766092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.766596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.766922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.767590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.772902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a721b5-e774-4d6b-be27-0e07fb5afcf1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.792338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1326/2645] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:39.798743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:39.815611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222308d4-d2be-46ce-b829-1dfbd51d912c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:39.821426 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:39.822927 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.823222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.823512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] PATH_INFO: `/v3/projects/c13f3b99aa0a41d98423f40ab14d56d3/users/96734957ee794fe38c6934f81f9c4f3e/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.851079 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13f3b99aa0a41d98423f40ab14d56d3, user_id=96734957ee794fe38c6934f81f9c4f3e, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:39.855300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:39.886447 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1320/2646] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:39 2022] PUT /identity/v3/projects/c13f3b99aa0a41d98423f40ab14d56d3/users/96734957ee794fe38c6934f81f9c4f3e/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:39.903179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:39.904271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:39.911317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:39.950370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] MFA Rules not processed for user `96734957ee794fe38c6934f81f9c4f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:39.951609 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:40.047062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1327/2647] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:40.054185 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.198519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec259-a336-40c1-bfe3-9b493e6bd626 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.208896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.210059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.210351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.210992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.211799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.216084 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.262361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1321/2648] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:40 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:40.272109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.280568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:40.287387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658dac6-4d88-4688-acd3-71a3971cb2fa None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.395526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.396999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.397495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.397952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.398870 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.401024 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.440774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1328/2649] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:40 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:40.443863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.464980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd29f6-5104-4efe-9558-82d14298b123 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.474850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.475489 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.475799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.476105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.476795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.479097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.499418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1322/2650] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:40.508053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.528120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed93d1-f9e8-428c-803b-8973aa67f34c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.535766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.536951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.537482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.538049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] PATH_INFO: `/v3/projects/f44da52bdd344d3885348c19ebff7e84/users/d421a4e946cf453fbea3a18f7df343f8/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.562950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44da52bdd344d3885348c19ebff7e84, user_id=d421a4e946cf453fbea3a18f7df343f8, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.565683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.584538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1329/2651] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:40 2022] PUT /identity/v3/projects/f44da52bdd344d3885348c19ebff7e84/users/d421a4e946cf453fbea3a18f7df343f8/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:40.589642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.609138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c376-a98f-4a53-b871-6fcdfcd24957 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.701384 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.702334 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.702901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.703485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.704786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.709193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cf4d8-8743-4360-a121-c72aca010b2f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.722677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:40.740139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:40.745674 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1323/2652] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:40.764950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc30d6-0701-49dd-9d0f-e6a8405a9e05 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:40.775187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:40.776896 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.777465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.778086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] PATH_INFO: `/v3/projects/f44da52bdd344d3885348c19ebff7e84/users/d421a4e946cf453fbea3a18f7df343f8/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.783032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44da52bdd344d3885348c19ebff7e84, user_id=d421a4e946cf453fbea3a18f7df343f8, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:40.788748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:40.815518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1330/2653] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:40 2022] PUT /identity/v3/projects/f44da52bdd344d3885348c19ebff7e84/users/d421a4e946cf453fbea3a18f7df343f8/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:40.835709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:40.836221 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:40.836601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:40.890353 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] MFA Rules not processed for user `d421a4e946cf453fbea3a18f7df343f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:40.892923 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:41.016073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1324/2654] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:41.035950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:41.133166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60fabc-f9dc-46e0-9cab-3d77e324de85 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:41.138006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:41.138643 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:41.138914 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:41.139188 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:41.152153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:41.154911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:41.172572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1331/2655] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:41.924537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:41.937207 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dfe54-08ab-4f9e-83ab-b8fb16a35a17 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:41.941074 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:41.941519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:41.941798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:41.942068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:42.034404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:42.037385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783209a-2a5e-41ca-baf3-8222d8e00420 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:42.057158 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1325/2656] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:42.415761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:42.417517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:42.575683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e44d55-29f6-445c-9009-624e4b419512 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:42.587004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:42.588152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:42.588783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:42.589374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] PATH_INFO: `/v3/users/b8ab0d2a5e1b4b4ab4378bd89d2db123` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:42.591675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ab0d2a5e1b4b4ab4378bd89d2db123)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:42.595868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:42.598985 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:42.695662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ab0d2a5e1b4b4ab4378bd89d2db123'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:42.707368 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ab0d2a5e1b4b4ab4378bd89d2db123'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:42.717252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ab0d2a5e1b4b4ab4378bd89d2db123'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:42.718240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ab0d2a5e1b4b4ab4378bd89d2db123'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:42.735703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ab0d2a5e1b4b4ab4378bd89d2db123'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:42.746201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1332/2657] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:42 2022] DELETE /identity/v3/users/b8ab0d2a5e1b4b4ab4378bd89d2db123 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:42.751796 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:42.752222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:42.752540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:42.783369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:42.784705 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:42.890787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:42.896046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:42.907316 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1326/2658] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:42.919044 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:42.919718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb264e-73d7-4c7e-9275-d4cc5ff0dbd0 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:42.922418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:42.929848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:42.930957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:42.933053 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:42.933495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] PATH_INFO: `/v3/users/e70207f0ac9845098851dfb248c7e91f` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:42.936542 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70207f0ac9845098851dfb248c7e91f)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:42.939971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:42.975912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:43.045402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23bf09-b3b6-4ea1-8fbd-d452183548e6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.053059 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.054549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.055014 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.056294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.057917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70207f0ac9845098851dfb248c7e91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.075470 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70207f0ac9845098851dfb248c7e91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.075694 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.077531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ca25-0436-4005-bacf-3ff7ce8e3704 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.093958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70207f0ac9845098851dfb248c7e91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.095202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70207f0ac9845098851dfb248c7e91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.096845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1327/2659] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:43.146289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70207f0ac9845098851dfb248c7e91f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.160901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1333/2660] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:42 2022] DELETE /identity/v3/users/e70207f0ac9845098851dfb248c7e91f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:43.167203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.168620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.169111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.213604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:43.215135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.304522 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1328/2661] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:43.319036 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.321734 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.379241 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.381369 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.411377 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b511c-7213-4e17-b3ed-a40b3709507f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.415289 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbee53d-fe11-4dd1-91e2-9aea248b4de3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.419601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.420340 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.420735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.421094 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] PATH_INFO: `/v3/projects/dbf312281cdc4ad4925d1815c3ac7881` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.421282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.422068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.422468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.422855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf312281cdc4ad4925d1815c3ac7881)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.423362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.425082 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.434372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:43.441284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.444671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c278c1e-9610-4914-aa0e-fd1d86ad04ab service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.458414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1334/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:43.507742 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf312281cdc4ad4925d1815c3ac7881', 'request_id': 'req-252bd815-43e8-4081-afeb-a34e0cd9ea79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.529571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf312281cdc4ad4925d1815c3ac7881', 'request_id': 'req-252bd815-43e8-4081-afeb-a34e0cd9ea79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.550202 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf312281cdc4ad4925d1815c3ac7881', 'request_id': 'req-252bd815-43e8-4081-afeb-a34e0cd9ea79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.571015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] The token cache is being invalidate because project dbf312281cdc4ad4925d1815c3ac7881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:43.571372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.571979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-252bd815-43e8-4081-afeb-a34e0cd9ea79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:43.577804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1329/2663] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:43 2022] DELETE /identity/v3/projects/dbf312281cdc4ad4925d1815c3ac7881 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:43.579973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.580336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.580692 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.607614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:43.609636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.717209 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1335/2664] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:43.727737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.728152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.728556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.761393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:43.763649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.775069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1330/2665] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:43.779779 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.790672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028c048-50bf-46c8-ad25-1e69e448daf5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.795595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.796226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.796507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.796959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.797627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:43.797934 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.799543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.815215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1336/2666] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:43.822011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.833913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d1340-98ef-465e-b973-a42b26ae87b6 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.840206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.841042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.841496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.841915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.842757 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.845148 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.861050 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.863987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:43.895632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0ef27-4b3d-4cc5-9233-36ea149ed5c1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:43.919172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1331/2667] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:43 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:43.919422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:43.921172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:43.921172 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:43.921793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] PATH_INFO: `/v3/projects/9bff2dbd28314ed5ac9852e9238eee3a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:43.924215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bff2dbd28314ed5ac9852e9238eee3a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:43.925424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:43.928439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:43.961832 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:44.003975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bff2dbd28314ed5ac9852e9238eee3a', 'request_id': 'req-9d9e2d67-5dbd-4c24-9242-51ba56796556'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.013748 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bff2dbd28314ed5ac9852e9238eee3a', 'request_id': 'req-9d9e2d67-5dbd-4c24-9242-51ba56796556'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.019798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7aa39-0200-4d15-a760-02e671b8061c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.037412 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bff2dbd28314ed5ac9852e9238eee3a', 'request_id': 'req-9d9e2d67-5dbd-4c24-9242-51ba56796556'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.055591 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] The token cache is being invalidate because project 9bff2dbd28314ed5ac9852e9238eee3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:44.055974 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.057044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.059347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1337/2668] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:43 2022] DELETE /identity/v3/projects/9bff2dbd28314ed5ac9852e9238eee3a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:44.064358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.066745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.110840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.111604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.111970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.112255 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.113551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.115646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.147782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e2d67-5dbd-4c24-9242-51ba56796556 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.151259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1332/2669] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:43 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:44.154569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.155381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.155662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.156155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] PATH_INFO: `/v3/users/0067d8b66756417a92e50ab88251b0ec` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.157237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0067d8b66756417a92e50ab88251b0ec)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.157675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.160405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.160623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.166894 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:44.184116 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ddec4-5dd6-4ebc-9b78-242c61376955 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.194080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.194958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.195452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.195957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.197416 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.200463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.219623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1333/2670] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:44.224336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.236231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eedde3-d99f-47a0-b94f-e3d4510df891 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.243652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.244267 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.244649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.244840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.245495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.247193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.254455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0067d8b66756417a92e50ab88251b0ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.262083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1334/2671] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:44.272921 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0067d8b66756417a92e50ab88251b0ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.275036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.282673 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0067d8b66756417a92e50ab88251b0ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.283508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0067d8b66756417a92e50ab88251b0ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.286751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a557ba-6d6d-4e91-8b7c-e8537d91ff2b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.293600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.294595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.295042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.295498 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] PATH_INFO: `/v3/projects/2d9fa722c18045678015c7b300ccc3ab/users/4a20bdc6eee841258bd6ac11a8095a4a/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.306627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4cf594c3-efaa-4186-b2ea-0336477557c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0067d8b66756417a92e50ab88251b0ec'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.320410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1338/2672] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:44 2022] DELETE /identity/v3/users/0067d8b66756417a92e50ab88251b0ec => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:44.325199 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9fa722c18045678015c7b300ccc3ab, user_id=4a20bdc6eee841258bd6ac11a8095a4a, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.330378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.350769 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1335/2673] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:44 2022] PUT /identity/v3/projects/2d9fa722c18045678015c7b300ccc3ab/users/4a20bdc6eee841258bd6ac11a8095a4a/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:44.355941 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.356225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.356497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.383108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:44.384991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.493508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1339/2674] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:44.505420 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.508087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.633307 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cc225-fa08-4633-8c10-c6834cf414c5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.638619 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.639380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.644712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.644712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.657481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.659792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be4e6d-15ac-408a-9a67-b8bba219475a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.680813 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1336/2675] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:44.719346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:44.721745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.763423 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418c227-3207-48fa-a4ee-e9a2c7b259b6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:44.769571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:44.770303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.770656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.771000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] PATH_INFO: `/v3/projects/1c872137d05f46a1a3676de7b5a01b91` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.772511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c872137d05f46a1a3676de7b5a01b91)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:44.774661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:44.875166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c872137d05f46a1a3676de7b5a01b91', 'request_id': 'req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.884183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c872137d05f46a1a3676de7b5a01b91', 'request_id': 'req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.895814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c872137d05f46a1a3676de7b5a01b91', 'request_id': 'req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.915761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] The token cache is being invalidate because project 1c872137d05f46a1a3676de7b5a01b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:44.916240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.917373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-68ae3860-6c4d-4bf7-bb5e-d1d6c3ae28ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:44.919884 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1340/2676] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:44 2022] DELETE /identity/v3/projects/1c872137d05f46a1a3676de7b5a01b91 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:41:44.931884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436236ec-5701-47f7-97d7-887915156992 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:44.932529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436236ec-5701-47f7-97d7-887915156992 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:44.933111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436236ec-5701-47f7-97d7-887915156992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:44.955907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-436236ec-5701-47f7-97d7-887915156992 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:44.958527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-436236ec-5701-47f7-97d7-887915156992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:44.975020 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:45.062279 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1337/2677] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:45.071832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.072334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.072907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.094677 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:45.103624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:45.105729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:45.115972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1341/2678] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:45.123090 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436236ec-5701-47f7-97d7-887915156992 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.140311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436236ec-5701-47f7-97d7-887915156992 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.146671 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.147647 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.148106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.148808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.149878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:45.150325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.153332 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.170634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1338/2679] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:45.176297 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.189660 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af7eb5-7646-4612-8222-66b424e574d6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.195958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.197754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.197754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.198139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.198880 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.201255 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.248361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1342/2680] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:45 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:45.250258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.267339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5425e-3679-4ca4-9d6c-31e16bff383b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.381996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.382657 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.383167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.383668 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.384876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.388856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.423613 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1339/2681] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:45 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:45.431243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.453736 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f8f2f-1fc7-455a-aff3-7f0d9a363c1e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.463639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.464759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.465508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.466009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.467353 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.470950 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.488777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1343/2682] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:45.494743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.506086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc8100-b114-464e-b1e2-d43c7c7be3cb admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.513974 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.515043 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.515541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.516035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.517208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.519980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6c4b3-0c98-4bce-825e-a49e6fca05af admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.545457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1340/2683] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:45.547549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.566821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb64834-360e-42f1-8cbd-10ca503c5e39 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.571776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.572457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.572783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.573045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] PATH_INFO: `/v3/projects/0a7b887ec12b41fabbaa4828544fcc4d/users/e235c85b19564bb3ad194b67465fd0bd/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.592835 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7b887ec12b41fabbaa4828544fcc4d, user_id=e235c85b19564bb3ad194b67465fd0bd, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:45.599854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:45.633202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1344/2684] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:45 2022] PUT /identity/v3/projects/0a7b887ec12b41fabbaa4828544fcc4d/users/e235c85b19564bb3ad194b67465fd0bd/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:45.653130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410473-cd98-48f8-bfda-ad136a207e85 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.653632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410473-cd98-48f8-bfda-ad136a207e85 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.654122 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410473-cd98-48f8-bfda-ad136a207e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:45.707773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0f410473-cd98-48f8-bfda-ad136a207e85 None None] MFA Rules not processed for user `e235c85b19564bb3ad194b67465fd0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:45.709843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0f410473-cd98-48f8-bfda-ad136a207e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:45.813620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1341/2685] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:45.833519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:45.836456 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:45.966968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544ea57-cc8e-4636-ab91-1ea78604c78d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:45.979051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:45.979703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:45.980251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:45.980251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:46.004166 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:46.008014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:46.032323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1345/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:51.183299 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:51.184166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:51.184166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:51.234603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:51.259662 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] MFA Rules not processed for user `4a20bdc6eee841258bd6ac11a8095a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:51.261208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:51.440950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1342/2687] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:51.446653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:51.450079 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:51.711987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d679-4608-4b14-8c0a-5908c0d94579 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:51.717861 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:51.718943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:51.719345 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:51.720819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:51.722569 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:51.726391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:51.815606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1346/2688] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:51 2022] POST /identity/v3/projects => generated 371 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:51.823182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:51.863843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6465c99-231c-40fb-854e-82c30a76281c None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:51.973589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:51.974388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:51.974745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:51.975153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:51.976459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:51.979549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:52.012541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1343/2689] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:51 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:52.017681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:52.036121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169af70-b21c-420b-923b-04b2510a6cb6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:52.069944 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:52.070897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:52.071206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:52.071658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:52.073586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:52.076620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:52.097918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1347/2690] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:52.105556 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:52.123690 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4b022-d8aa-4a4d-aee3-ac4d1627defc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:52.137234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:52.138853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:52.139254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:52.139735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] PATH_INFO: `/v3/projects/f5212b0ef07c4ad187500cffc36ff931/users/376759e72d214aeab86b4f605b913154/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:52.161248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5212b0ef07c4ad187500cffc36ff931, user_id=376759e72d214aeab86b4f605b913154, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:52.163981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf457f6-3115-4279-b9e9-5b1cd8885f8d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:52.187726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1344/2691] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:52 2022] PUT /identity/v3/projects/f5212b0ef07c4ad187500cffc36ff931/users/376759e72d214aeab86b4f605b913154/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:52.194246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:52.219275 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc61bf-62a9-40e7-9cf3-511d3d46f73d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:52.244429 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:52.244911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:52.245194 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:52.258208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:52.279026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] MFA Rules not processed for user `d882c87d5de7497b822ec7b1798d2008`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:52.280844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:52.348284 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:52.348968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:52.349249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:52.349523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:52.350174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:52.353305 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:52.383952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1348/2692] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:41:52.395245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:52.417965 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469087f-bdb5-42a4-923e-8dd5db565a8a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:52.428771 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:52.430222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:52.430767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:52.431155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] PATH_INFO: `/v3/projects/f5212b0ef07c4ad187500cffc36ff931/users/376759e72d214aeab86b4f605b913154/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:52.433860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5212b0ef07c4ad187500cffc36ff931, user_id=376759e72d214aeab86b4f605b913154, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:52.437837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:52.472018 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1349/2693] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:52 2022] PUT /identity/v3/projects/f5212b0ef07c4ad187500cffc36ff931/users/376759e72d214aeab86b4f605b913154/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:52.489015 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1345/2694] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:53.953267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:54.107362 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:54.118161 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59fa1-f991-4490-b7f0-c14adff914a6 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:54.125063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:54.125999 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:54.126471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:54.126905 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] PATH_INFO: `/v3/users/e235c85b19564bb3ad194b67465fd0bd` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:54.130458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e235c85b19564bb3ad194b67465fd0bd)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:54.134278 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:54.154643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:54.276882 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e235c85b19564bb3ad194b67465fd0bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:54.278636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d337f8-02f1-42c9-bfa5-84fbf16c4ba0 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:54.328350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e235c85b19564bb3ad194b67465fd0bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:54.350907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e235c85b19564bb3ad194b67465fd0bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:54.352521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e235c85b19564bb3ad194b67465fd0bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:54.405135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-91842e80-8bf4-4aec-9248-0142f3d1291b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e235c85b19564bb3ad194b67465fd0bd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:54.417830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1350/2695] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:53 2022] DELETE /identity/v3/users/e235c85b19564bb3ad194b67465fd0bd => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:54.424563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:54.426323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:54.427322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:54.451514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:54.453223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:54.470909 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:54.473210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:54.476254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:54.477253 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:54.484103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:54.494522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:54.632929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1351/2696] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:54.645459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:54.649063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:54.651418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:54.651604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:54.693297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1346/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:54.795970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee33fc-726a-48ec-9132-11a504110c35 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:54.806621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:54.808571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:54.809308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:54.810739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:54.836484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:54.842220 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8e4f2-8a8c-4d79-b26c-15083fb1e791 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:54.896394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1352/2698] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:41:55.718227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:55.723259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:55.782661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef55e0-3972-4741-acac-6075ea9e71e4 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:55.798019 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:55.799223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:55.799953 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:55.800540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] PATH_INFO: `/v3/projects/0a7b887ec12b41fabbaa4828544fcc4d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:55.802559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7b887ec12b41fabbaa4828544fcc4d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:55.806595 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:55.921013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7b887ec12b41fabbaa4828544fcc4d', 'request_id': 'req-10aa10e7-637c-484b-a114-675153a34b41'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:55.938520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7b887ec12b41fabbaa4828544fcc4d', 'request_id': 'req-10aa10e7-637c-484b-a114-675153a34b41'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:55.954913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7b887ec12b41fabbaa4828544fcc4d', 'request_id': 'req-10aa10e7-637c-484b-a114-675153a34b41'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:56.027816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] The token cache is being invalidate because project 0a7b887ec12b41fabbaa4828544fcc4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:41:56.027816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:56.034078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-10aa10e7-637c-484b-a114-675153a34b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:56.043530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1347/2699] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:41:55 2022] DELETE /identity/v3/projects/0a7b887ec12b41fabbaa4828544fcc4d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:56.046328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:56.046877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:56.047179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:56.115240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:56.117304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:56.315897 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1353/2700] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:56.329087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:56.329414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:56.329896 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:56.357535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:56.407519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:56.410363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:56.441694 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:56.445505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1348/2701] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:41:56.460151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c962ba-69de-414a-be88-f06bd3b7d809 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:56.483675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:56.486017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:56.486867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:56.488444 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:56.489555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:41:56.490703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:56.494141 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:56.529405 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1354/2702] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:41:56.539313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:56.591709 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e0f4-6f23-4688-a85d-2a75e7535cad None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:56.615439 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:56.616696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:56.616696 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:56.617033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:56.617926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:56.623889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:56.722046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:56.722484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1349/2703] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:41:56 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:41:56.746454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ed132-b7ab-4618-a651-5e2b7adfc344 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:57.008165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:57.109263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:57.117915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:57.121678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:57.124528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:57.127596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:57.137607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:57.235586 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1355/2704] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:41:56 2022] POST /identity/v3/users => generated 292 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:41:57.240752 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:57.311367 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0ccac-fbc0-426d-93ba-ff7fb2e3e6ed admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:57.329177 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:57.330306 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:57.330840 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:57.331364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:57.340262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:57.353147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:57.380448 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1350/2705] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:57.387351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:57.411397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c234d7-7043-4cc3-9fb1-b004cd9f39b4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:57.421757 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:57.426265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:57.426783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:57.427279 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:57.428521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:57.431808 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850fc4b-d521-4720-ae82-6d42a7b47513 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:57.466577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:57.467785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1356/2706] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:41:57.488123 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f750185-f49a-4669-88a9-f73f04e6c29f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:57.501814 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:57.503010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:57.503478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:57.504150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] PATH_INFO: `/v3/projects/d05e27bd70e64afbbc58379e64826217/users/eb53ae825eea452d9d456233ac2af43c/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:57.553508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05e27bd70e64afbbc58379e64826217, user_id=eb53ae825eea452d9d456233ac2af43c, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:57.560385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:57.622894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1351/2707] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:41:57 2022] PUT /identity/v3/projects/d05e27bd70e64afbbc58379e64826217/users/eb53ae825eea452d9d456233ac2af43c/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:57.624153 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:57.624865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:57.625328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:57.652299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:41:57.654488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:57.806051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1357/2708] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:41:57.824242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:57.826940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:58.006548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e2801-ded8-4580-95f7-9e30bd027134 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:58.011849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:58.012845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:58.013342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:58.013869 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:58.040211 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:58.045246 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66edf97-3821-4377-8c86-ef4280afd652 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:58.087399 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1352/2709] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:41:59.737973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:41:59.764671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:41:59.804574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedda64-4e8c-4d62-a10b-4d23a077e9a2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:41:59.814409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:41:59.814874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:41:59.819630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:41:59.820324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] PATH_INFO: `/v3/users/9aa4240e77854e8a9fec966217a5bda0` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:41:59.825370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa4240e77854e8a9fec966217a5bda0)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:41:59.832247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:41:59.954103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa4240e77854e8a9fec966217a5bda0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:59.968130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa4240e77854e8a9fec966217a5bda0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:59.990653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa4240e77854e8a9fec966217a5bda0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:41:59.992068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa4240e77854e8a9fec966217a5bda0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:00.048825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d810720d-0cdc-4ee7-b5b8-4e6952c8de69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa4240e77854e8a9fec966217a5bda0'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:00.076797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1358/2710] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:41:59 2022] DELETE /identity/v3/users/9aa4240e77854e8a9fec966217a5bda0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:00.085871 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:00.087347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:00.089603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:00.155297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:00.157508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:00.440693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:00.441292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:00.441292 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:00.460992 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1353/2711] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:00.491876 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] MFA Rules not processed for user `eb53ae825eea452d9d456233ac2af43c`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:00.495128 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:00.495863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:00.498904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:00.618391 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:00.703149 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:00.739981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476f5f1-f7a7-4514-9010-a5593aeb3c01 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:00.752330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1359/2712] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:00.758289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:00.759355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:00.759861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:00.760343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:00.771968 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:00.778669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:00.795013 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:00.797802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:00.837167 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1354/2713] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:01.069407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b671a86-6b51-4e31-aee9-09aa5b18e5c6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:01.080394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:01.081493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:01.081970 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:01.082446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:01.123874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:01.131954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:01.182982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1360/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:02.266793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:02.270698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:02.326229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373888f-490b-401b-a35d-a3825f337c0e service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:02.364967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:02.366010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:02.366726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:02.370854 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] PATH_INFO: `/v3/projects/0a9ebc9dbe8145639975b429baa99cc5` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:02.373894 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9ebc9dbe8145639975b429baa99cc5)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:02.381488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:02.561325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9ebc9dbe8145639975b429baa99cc5', 'request_id': 'req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:02.584821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9ebc9dbe8145639975b429baa99cc5', 'request_id': 'req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:02.606637 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9ebc9dbe8145639975b429baa99cc5', 'request_id': 'req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:02.632007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] The token cache is being invalidate because project 0a9ebc9dbe8145639975b429baa99cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:02.632411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:02.635163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-4eebbb0e-07bb-47a6-af65-a814ce8ed5dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:02.639926 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1355/2715] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:02 2022] DELETE /identity/v3/projects/0a9ebc9dbe8145639975b429baa99cc5 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:02.647118 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:02.681351 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecca46-4c6e-4d21-ba18-ebf8420dc5a3 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:02.916281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:02.917192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:02.917558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:02.917987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] PATH_INFO: `/v3/users/96734957ee794fe38c6934f81f9c4f3e` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:02.919635 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96734957ee794fe38c6934f81f9c4f3e)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:02.923203 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:02.998245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:02.998548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:02.999010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:03.077220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] MFA Rules not processed for user `eb53ae825eea452d9d456233ac2af43c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:03.079235 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:03.103848 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96734957ee794fe38c6934f81f9c4f3e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:03.130893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96734957ee794fe38c6934f81f9c4f3e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:03.164494 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96734957ee794fe38c6934f81f9c4f3e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:03.165824 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96734957ee794fe38c6934f81f9c4f3e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:03.192149 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:03.195457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96734957ee794fe38c6934f81f9c4f3e'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:03.208624 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1361/2716] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:02 2022] DELETE /identity/v3/users/96734957ee794fe38c6934f81f9c4f3e => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:03.326600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1356/2717] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:04.126893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:04.129765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:04.547952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e94b19-e0c8-4620-a39a-37be2fd6d29d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:04.554545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:04.555455 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:04.555738 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:04.556218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] PATH_INFO: `/v3/projects/c13f3b99aa0a41d98423f40ab14d56d3` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:04.558512 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13f3b99aa0a41d98423f40ab14d56d3)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:04.560777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:04.640090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13f3b99aa0a41d98423f40ab14d56d3', 'request_id': 'req-913f271d-e3e6-4f95-b892-69044b297ff4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.656798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13f3b99aa0a41d98423f40ab14d56d3', 'request_id': 'req-913f271d-e3e6-4f95-b892-69044b297ff4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.673895 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13f3b99aa0a41d98423f40ab14d56d3', 'request_id': 'req-913f271d-e3e6-4f95-b892-69044b297ff4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.695454 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] The token cache is being invalidate because project c13f3b99aa0a41d98423f40ab14d56d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:04.696148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.697466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-913f271d-e3e6-4f95-b892-69044b297ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.705521 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1362/2718] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:04 2022] DELETE /identity/v3/projects/c13f3b99aa0a41d98423f40ab14d56d3 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:04.707139 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:04.722865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc3a2c-c297-426d-a9fc-6bcba0f98860 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:04.809736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:04.832477 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:04.832931 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:04.833217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:04.833502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] PATH_INFO: `/v3/users/d421a4e946cf453fbea3a18f7df343f8` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:04.835035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d421a4e946cf453fbea3a18f7df343f8)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:04.838943 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:04.952140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd421a4e946cf453fbea3a18f7df343f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.970223 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd421a4e946cf453fbea3a18f7df343f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.981649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd421a4e946cf453fbea3a18f7df343f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.983101 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd421a4e946cf453fbea3a18f7df343f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:04.984222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd421a4e946cf453fbea3a18f7df343f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:05.017193 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1357/2719] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:04 2022] DELETE /identity/v3/users/d421a4e946cf453fbea3a18f7df343f8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:05.914040 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:05.914563 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:05.914669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:06.002658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] MFA Rules not processed for user `376759e72d214aeab86b4f605b913154`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:06.005527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:06.067762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:06.290403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1363/2720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:06.434843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:06.437496 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:06.500183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:06.503438 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:06.728659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19e4b5-d356-4f34-927a-b845d43989da admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:06.738812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:06.740441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:06.741297 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:06.742147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] PATH_INFO: `/v3/users/b5ada17b384640ab968610cacb8b7f79` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:06.744944 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ada17b384640ab968610cacb8b7f79)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:06.750639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:06.772311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283d0bf-c2d8-4e75-b0a3-3a65c03d7993 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:06.779538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:06.780237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:06.780656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:06.780847 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] PATH_INFO: `/v3/projects/f44da52bdd344d3885348c19ebff7e84` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:06.781997 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44da52bdd344d3885348c19ebff7e84)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:06.784089 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:06.880514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ada17b384640ab968610cacb8b7f79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.903800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ada17b384640ab968610cacb8b7f79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.925065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:06.930806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ada17b384640ab968610cacb8b7f79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.934078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ada17b384640ab968610cacb8b7f79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.935463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ada17b384640ab968610cacb8b7f79'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.938457 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44da52bdd344d3885348c19ebff7e84', 'request_id': 'req-233356b5-20db-45b4-a323-50628fffaca4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.955815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44da52bdd344d3885348c19ebff7e84', 'request_id': 'req-233356b5-20db-45b4-a323-50628fffaca4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.967872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44da52bdd344d3885348c19ebff7e84', 'request_id': 'req-233356b5-20db-45b4-a323-50628fffaca4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.975398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1358/2721] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:06 2022] DELETE /identity/v3/users/b5ada17b384640ab968610cacb8b7f79 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:06.977342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:06.980531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:06.989429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] The token cache is being invalidate because project f44da52bdd344d3885348c19ebff7e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:06.990404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.991453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-233356b5-20db-45b4-a323-50628fffaca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:06.993820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1364/2722] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:06 2022] DELETE /identity/v3/projects/f44da52bdd344d3885348c19ebff7e84 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:06.996064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941087-ef48-4b1e-aacd-48c43e26f398 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:06.996640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941087-ef48-4b1e-aacd-48c43e26f398 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:06.997142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed941087-ef48-4b1e-aacd-48c43e26f398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.060683 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-ed941087-ef48-4b1e-aacd-48c43e26f398 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:07.062632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ed941087-ef48-4b1e-aacd-48c43e26f398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:07.166627 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:07.236082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1365/2723] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:07.243523 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:07.244427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:07.246193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.384903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9772e-895a-4579-91f0-57539a3b04af admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:07.396552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:07.398009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:07.398502 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:07.399005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.403576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:07.431699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:07.434228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:07.468958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1366/2724] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:07.473127 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:07.477789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:07.615372 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:07.623458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d373ad-851f-4909-9d76-8df762e76e7c service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:07.667809 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1359/2725] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:07.670480 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:07.670957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:07.671893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.709082 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a3ad2-80f5-4885-b399-02918bca457b None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:07.722531 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:07.725890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:07.727527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:07.729176 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.748786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:07.752233 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:07.762058 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:07.768763 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:07.776325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1360/2726] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:07.782609 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:07.816840 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1367/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:07.879544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d09a66-3ca5-41d8-b820-586cb5577ef5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:07.939996 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:07.943935 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:07.944454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:07.945550 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:07.950228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:07.950929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:07.957990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.019360 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1361/2728] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:42:08.025038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.045797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff4fb5-124b-47ef-a90c-c07f9264c8d8 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.055973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:08.057031 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.057490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.058027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:08.059369 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:08.063700 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.129810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1368/2729] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:08 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:42:08.134695 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.155895 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c92a01-7f44-45ad-9f2f-b991d9358f3f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.206942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:08.343184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:08.345516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.346773 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.350245 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:08.351904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:08.364845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.442254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1362/2730] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:08 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:08.448881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.494192 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3a77c-ba30-4a73-9efa-0544252b1e70 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.509742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:08.511071 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.511497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.511871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:08.512675 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:08.515887 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.531544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1369/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:08.538963 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.558744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb878da-3eed-44ef-a557-f704828ff820 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.568035 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:08.569147 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.569636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.570114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:08.571945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:08.575011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.605442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1363/2732] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:08.605761 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.653350 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890357fa-5ae7-46c5-a5fa-e5418e7a7781 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.719798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:08.722334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.726001 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.726854 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] PATH_INFO: `/v3/projects/f397b33ebadc48029803c116a353b456/users/51390ac2d1684be8bff220523b872e8a/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:08.738562 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:08.742249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:08.778628 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f397b33ebadc48029803c116a353b456, user_id=51390ac2d1684be8bff220523b872e8a, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:08.791704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931582b4-0438-4895-8e79-b485c28b2387 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:08.808018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ba3de-10fe-47ab-9d0a-137360b869e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:08.833228 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1370/2733] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:08 2022] PUT /identity/v3/projects/f397b33ebadc48029803c116a353b456/users/51390ac2d1684be8bff220523b872e8a/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:08.871619 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:08.872485 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:08.882070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:09.010151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] MFA Rules not processed for user `51390ac2d1684be8bff220523b872e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:09.017285 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:09.104572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:09.105807 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:09.106350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:09.107032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] PATH_INFO: `/v3/projects/f63f7676490e42029f07413387b9f08a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:09.112350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63f7676490e42029f07413387b9f08a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:09.120182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:09.201104 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:09.283437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1371/2734] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:09.295599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:09.298704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:09.302589 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63f7676490e42029f07413387b9f08a', 'request_id': 'req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.324285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63f7676490e42029f07413387b9f08a', 'request_id': 'req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.352249 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63f7676490e42029f07413387b9f08a', 'request_id': 'req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.384576 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] The token cache is being invalidate because project f63f7676490e42029f07413387b9f08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:09.385751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.387322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.392063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1364/2735] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:08 2022] DELETE /identity/v3/projects/f63f7676490e42029f07413387b9f08a => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:09.398242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:09.404385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:09.518762 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddafa3-8275-49a5-9b12-a09144b819b6 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:09.544383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:09.646806 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b6041-9ee6-43ea-a54f-c84b8f2deb4f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:09.655501 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:09.656383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:09.657259 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:09.657763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] PATH_INFO: `/v3/users/a44748ca35584878984af98fc37007f7` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:09.660010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44748ca35584878984af98fc37007f7)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:09.662531 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:09.686798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:09.688020 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:09.688236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:09.688739 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:09.695580 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:09.839978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44748ca35584878984af98fc37007f7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.871574 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:09.873587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:09.888152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1372/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:09.925743 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44748ca35584878984af98fc37007f7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.951917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:09.953947 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:09.961181 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44748ca35584878984af98fc37007f7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.962724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44748ca35584878984af98fc37007f7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.963956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd383251-f370-4feb-aebf-a44fe4706e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44748ca35584878984af98fc37007f7'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:09.984769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3930-bbab-4a27-a4db-de7924d7d4ae service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:09.990416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:09.992017 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:09.992407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:09.992774 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] PATH_INFO: `/v3/users/51390ac2d1684be8bff220523b872e8a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:09.994294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51390ac2d1684be8bff220523b872e8a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:09.997328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:10.013343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1365/2737] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:09 2022] DELETE /identity/v3/users/a44748ca35584878984af98fc37007f7 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:10.094587 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51390ac2d1684be8bff220523b872e8a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:10.100963 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:10.108507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51390ac2d1684be8bff220523b872e8a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:10.155418 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51390ac2d1684be8bff220523b872e8a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:10.159276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51390ac2d1684be8bff220523b872e8a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:10.256095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51390ac2d1684be8bff220523b872e8a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:10.281634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1373/2738] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:09 2022] DELETE /identity/v3/users/51390ac2d1684be8bff220523b872e8a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:10.289639 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:10.290537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:10.291280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:10.368921 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:10.371007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:10.553656 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1366/2739] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:10.576266 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:10.579208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:10.722007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79db88c-9840-4753-8408-73e36004a1e4 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:10.730909 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:10.732208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:10.732814 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:10.733298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:10.756773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:10.759528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:10.785665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1374/2740] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:11.516136 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:11.522473 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:11.608262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdf7e2-82ad-4328-b9f7-55478264d3d1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:11.632262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:11.634108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:11.634409 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:11.634702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] PATH_INFO: `/v3/projects/361a393f005e40679385fe26d5967781` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:11.641006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=361a393f005e40679385fe26d5967781)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:11.647779 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:11.756552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:11.759976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:11.803063 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a94b1e-67d0-4852-9c04-9fcc7b3785e3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:11.806956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:11.827950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361a393f005e40679385fe26d5967781', 'request_id': 'req-751bbb30-9be4-4e13-8041-9ef976820493'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:11.876309 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361a393f005e40679385fe26d5967781', 'request_id': 'req-751bbb30-9be4-4e13-8041-9ef976820493'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:11.904633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361a393f005e40679385fe26d5967781', 'request_id': 'req-751bbb30-9be4-4e13-8041-9ef976820493'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:11.942430 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] The token cache is being invalidate because project 361a393f005e40679385fe26d5967781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:11.942824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:11.957465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-751bbb30-9be4-4e13-8041-9ef976820493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:11.968691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1367/2741] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:11 2022] DELETE /identity/v3/projects/361a393f005e40679385fe26d5967781 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:11.972488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ba0-6190-481f-b685-9ea45cb020a1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:11.973481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ba0-6190-481f-b685-9ea45cb020a1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:11.974236 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69720ba0-6190-481f-b685-9ea45cb020a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:11.997956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:11.999283 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.000010 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.000464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] PATH_INFO: `/v3/projects/f397b33ebadc48029803c116a353b456` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.003634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f397b33ebadc48029803c116a353b456)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.015117 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.023423 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-69720ba0-6190-481f-b685-9ea45cb020a1 None None] MFA Rules not processed for user `4a20bdc6eee841258bd6ac11a8095a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:12.025511 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-69720ba0-6190-481f-b685-9ea45cb020a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:12.168321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f397b33ebadc48029803c116a353b456', 'request_id': 'req-87535169-7dda-4e45-ac56-f104f0289d79'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:12.181773 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f397b33ebadc48029803c116a353b456', 'request_id': 'req-87535169-7dda-4e45-ac56-f104f0289d79'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:12.199409 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f397b33ebadc48029803c116a353b456', 'request_id': 'req-87535169-7dda-4e45-ac56-f104f0289d79'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:12.244681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] The token cache is being invalidate because project f397b33ebadc48029803c116a353b456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:12.245338 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:12.246390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-87535169-7dda-4e45-ac56-f104f0289d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:12.257373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1368/2743] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:12.257373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf9542-7c56-486c-86ca-c493ea0c324b None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.257373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf9542-7c56-486c-86ca-c493ea0c324b None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.257373 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf9542-7c56-486c-86ca-c493ea0c324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.258140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1375/2743] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:11 2022] DELETE /identity/v3/projects/f397b33ebadc48029803c116a353b456 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:12.258140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254cc5-9211-4f9f-9de9-848434bed152 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.258140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254cc5-9211-4f9f-9de9-848434bed152 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.258140 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254cc5-9211-4f9f-9de9-848434bed152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.279182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:12.286978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-35cf9542-7c56-486c-86ca-c493ea0c324b None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:12.288794 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-35cf9542-7c56-486c-86ca-c493ea0c324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:12.328553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-53254cc5-9211-4f9f-9de9-848434bed152 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:12.332044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-53254cc5-9211-4f9f-9de9-848434bed152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:12.434257 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1369/2744] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:12.461519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.461519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.461824 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.510339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:12.512096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:12.523229 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1370/2745] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:12.528006 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.543855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26e8b7-d184-4ef2-8eed-34a8be14eea9 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:12.552845 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:12.554342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.555551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.557011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.558405 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:12.559119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.561654 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.569806 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1376/2746] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:12.584671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.585179 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.585662 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.593215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1371/2747] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:42:12.594726 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.613990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea488d9f-98fc-4f0c-a63b-bd59cf2bbdd2 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:12.641584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:12.642557 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.643295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.643899 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.645478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.652293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.680863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:12.683324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:12.720407 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1377/2748] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:12.726372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.728419 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1372/2749] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:42:12.733064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.748953 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76a8b6-3d22-45b4-846c-4fca0cca72f8 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:12.809505 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:12.854348 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cef3af-22b4-40a8-9958-78273486e152 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:12.867580 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:12.868642 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.868853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.869504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.870016 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.872551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.909339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1373/2750] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/users => generated 281 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:42:12.921404 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.923410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:12.925511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.925511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.926035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.926961 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:12.927406 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.929373 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.939581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b7292-0d16-4fd9-ba60-5eae0067f4ca admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:12.948951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1378/2751] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:42:12.951857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:12.953838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:12.954596 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:12.954789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:12.956621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:12.958111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:12.962995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:12.993919 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6e8dd-deae-4773-90ed-344b0189f05d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.003271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1374/2752] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:13.011453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.011867 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.014431 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.014750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.015666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.017699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.023424 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.033252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c56d3-b10a-45b0-bd7a-c26b4a551410 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.049747 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.051011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.052553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.054343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.057949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.063620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.110797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1375/2753] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:13.112719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.140075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1379/2754] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:12 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:42:13.140570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2526fc-2dc7-4048-a5e7-ca604b5705b5 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.148182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.321240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.321240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.321240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.321240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] PATH_INFO: `/v3/projects/810d78bbbbc64e12b8a5685a95d748cd/users/02232854adb247bfb981e61dce50d44a/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.357065 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810d78bbbbc64e12b8a5685a95d748cd, user_id=02232854adb247bfb981e61dce50d44a, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.361815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b422df6-64dc-400e-bcea-3ce3c4a40520 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.398942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1376/2755] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:13 2022] PUT /identity/v3/projects/810d78bbbbc64e12b8a5685a95d748cd/users/02232854adb247bfb981e61dce50d44a/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:13.411321 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.411759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.412048 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.415013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acfb97-fdc2-4ed1-ad59-73ab48238f6d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.462745 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:13.573625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.574633 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.575187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.575746 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.576991 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.581029 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.632462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1380/2756] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:13 2022] POST /identity/v3/users => generated 288 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:13.634764 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.655358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26758f87-e77b-4782-a73c-5e70e8d443b3 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.663952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.669191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.669903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.670582 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.671807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.677604 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.737687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1381/2757] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:13.742282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.749924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] MFA Rules not processed for user `02232854adb247bfb981e61dce50d44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:13.751219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:13.807722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37362181-e0d8-4814-a19a-34625cd7b4d1 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.823358 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.826221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.827590 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.828081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.829263 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.832250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.872922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1382/2758] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:13.872922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.876191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:13.977578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1377/2759] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:13.978073 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:14.018293 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de6d2-5f31-41a0-8703-37197ca466ae None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.037691 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.039300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.040088 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.040873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] PATH_INFO: `/v3/projects/e0aadf5390e64d64bf16c4cb283aadca/users/184d916641594820a3556ac4999c1060/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.072121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0aadf5390e64d64bf16c4cb283aadca, user_id=184d916641594820a3556ac4999c1060, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:14.076964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:14.081008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:14.102389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1378/2760] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:13 2022] PUT /identity/v3/projects/e0aadf5390e64d64bf16c4cb283aadca/users/184d916641594820a3556ac4999c1060/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:14.118326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:14.121889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.178075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6d036-efa0-4ff1-9f87-97b5c30a4531 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.249912 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2a79-b065-4ef4-a083-fb403cba913f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.257922 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.258958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.259413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.259893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.311213 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.312402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.313041 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.313639 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.317938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.350980 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:14.357371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:14.370221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237bf33-a939-4acd-958c-9b3ee70e80c8 service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:14.402478 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:14.405520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:14.410168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1383/2761] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:14.412270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.416550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.417468 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.432004 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1379/2762] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:14.477777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:14.511602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] MFA Rules not processed for user `184d916641594820a3556ac4999c1060`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:14.513816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.684402 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616fe0-1012-49e4-a641-17fa596a2d28 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.691553 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.693011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.693592 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.694161 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] PATH_INFO: `/v3/users/02232854adb247bfb981e61dce50d44a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.698670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02232854adb247bfb981e61dce50d44a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:14.698670 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:14.737815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1384/2763] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:14.759783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:14.764828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.835394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02232854adb247bfb981e61dce50d44a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:14.841326 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:14.891327 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02232854adb247bfb981e61dce50d44a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:14.941469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925183bd-9633-4b22-bfd8-925a124b6cef None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.953810 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02232854adb247bfb981e61dce50d44a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:14.954683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02232854adb247bfb981e61dce50d44a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:14.955701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-3623ea77-ba0c-4626-a3cf-cd2e01895862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02232854adb247bfb981e61dce50d44a'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:14.958695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.961630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.962047 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.962328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.976385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.994763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1380/2764] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:14 2022] DELETE /identity/v3/users/02232854adb247bfb981e61dce50d44a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:15.001009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:15.001681 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:15.003460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:15.011596 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:15.014469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:15.052768 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:15.056502 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1385/2765] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:15.056853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:15.279220 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1381/2766] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:15.304898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:15.306681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:15.489402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676df5-62d1-46a8-a9ce-081f55424664 service glance] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:15.676320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=100 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:15.705387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:15.706270 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:15.706461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:15.706707 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:15.722793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:15.725101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a5155-119b-4af3-9efb-02e9cfb93b10 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:15.757967 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1386/2767] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:16.298457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:16.300760 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:16.326133 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45317-0a5b-462f-923b-e8af175723c5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:16.339546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:16.341215 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:16.342227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:16.342907 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] PATH_INFO: `/v3/projects/810d78bbbbc64e12b8a5685a95d748cd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:16.345706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=810d78bbbbc64e12b8a5685a95d748cd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:16.350022 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:16.456521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810d78bbbbc64e12b8a5685a95d748cd', 'request_id': 'req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:16.485383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810d78bbbbc64e12b8a5685a95d748cd', 'request_id': 'req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:16.497754 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810d78bbbbc64e12b8a5685a95d748cd', 'request_id': 'req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:16.522865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] The token cache is being invalidate because project 810d78bbbbc64e12b8a5685a95d748cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:16.523208 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:16.524010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-ee8c4fbd-48c6-45ff-8d23-aa2c8a6025bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:16.526975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1382/2768] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:16 2022] DELETE /identity/v3/projects/810d78bbbbc64e12b8a5685a95d748cd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:16.534825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:16.535719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:16.535719 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:16.567687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:16.569703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:16.906100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1387/2769] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:16.917804 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:16.918948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:16.919878 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.005344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:17.009504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:17.036712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1383/2770] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:17.042165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.064198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b43b8a-a605-46a6-b74c-eac133b4e7b2 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.096845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.098432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.099324 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.100224 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.102095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:17.103821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.108481 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.151484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1388/2771] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:42:17.159661 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.178967 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a151a-b412-49bc-827c-e2f83a6a1fe1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.186751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.187660 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.188128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.188470 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.189421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.192069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.251455 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1384/2772] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:17 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:17.257365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.315881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95ac33-ac73-470b-998a-f48c82d55f97 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.511506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.513823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.515654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.516318 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.518155 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.521720 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.565378 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1389/2773] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:17 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:17.568258 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.615712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5d27-8e32-4662-adb3-aeb5fe0afdb6 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.625523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.626566 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.627100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.627704 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.629318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.632817 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.657512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1385/2774] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:17.663375 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.704044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813da8-3a6b-41f5-aa0c-0b9cd7edc883 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.732327 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.734525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.735473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.736265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.738362 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.742150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48fa6c5-9290-4aef-989d-ad79e2e678f3 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.770504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1390/2775] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:17.776983 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.799518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b195c58-a3e8-4387-8ea7-e0aedb705585 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.819815 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.824380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.825160 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.827018 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] PATH_INFO: `/v3/projects/8d170930e8c04a52be39723909f9ef94/users/a6bed078c4fe4016a0ecfcee6b62cbbe/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.854176 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:17.901636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d170930e8c04a52be39723909f9ef94, user_id=a6bed078c4fe4016a0ecfcee6b62cbbe, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.913339 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.956736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1386/2776] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:17 2022] PUT /identity/v3/projects/8d170930e8c04a52be39723909f9ef94/users/a6bed078c4fe4016a0ecfcee6b62cbbe/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:17.962540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.962878 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.963175 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.020718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] MFA Rules not processed for user `a6bed078c4fe4016a0ecfcee6b62cbbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:18.024204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:18.126309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:18.199630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1391/2777] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:18.205720 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:18.357319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fd1c5-c915-4db3-bda3-aa2c0ce49bc4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:18.364863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:18.365751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.366030 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.366382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.367025 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:18.369717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:18.428541 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1387/2778] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:18 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:18.432816 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:18.452776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9eade-947c-4667-af3e-e1350b8bedfc None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:18.580678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:18.581786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.581786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.581786 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.582361 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:18.584320 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:18.615568 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1392/2779] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:18 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:18.621822 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:18.637023 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405422-ff37-4109-8726-ef197891e01c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:18.650292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:18.650938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.651219 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.651504 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.652164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:18.653916 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:18.689607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1388/2780] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:18.700080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:18.723200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29604134-446c-4be7-a9d9-3b87acaac2c7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:18.731296 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:18.732167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.732696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.733182 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.734464 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:18.737435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba894-d94a-4574-ba60-c36cd1d51bb9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:18.755516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1393/2781] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:18.762551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:18.779904 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925721b-208e-4dc0-b284-9c8d18dad3c1 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:18.789113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:18.790234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.790707 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.791188 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] PATH_INFO: `/v3/projects/9fc82c4a8a6e418d8b77c0da88f372ce/users/517dbc84dafa4349b904c6a7bfd794bb/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.815002 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc82c4a8a6e418d8b77c0da88f372ce, user_id=517dbc84dafa4349b904c6a7bfd794bb, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:18.819057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:18.838458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1389/2782] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:18 2022] PUT /identity/v3/projects/9fc82c4a8a6e418d8b77c0da88f372ce/users/517dbc84dafa4349b904c6a7bfd794bb/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:18.860647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:18.860647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:18.861546 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:18.930208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] MFA Rules not processed for user `517dbc84dafa4349b904c6a7bfd794bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:18.932735 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:19.066545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1394/2783] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:19.072940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.191365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699411f-0301-4619-8c73-5a0ec635e97d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.198406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.210153 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.211329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.212135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.213648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.216858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:19.280343 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1390/2784] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:19 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:19.288217 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.328515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba52480-31ac-4492-a8b2-d0bc79ad8fa0 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.432009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:19.450139 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.451168 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.451695 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.452165 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.455756 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.459208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:19.508722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1395/2785] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:19 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:42:19.514163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.533390 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a92553-6c18-4e15-803a-0459bba245e8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.543270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.544310 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.544865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.546682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.548886 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.552093 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:19.575549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1391/2786] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:19.582782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.605901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30b601-1536-4e57-aa84-4e6e8aa0bf76 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.618741 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.623594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.623594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.623594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] PATH_INFO: `/v3/projects/40dc0feee9574c88b6988fcaf607ab0c/users/9e5e965ac8ca4a1d93886d722ff0b2cb/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.650572 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc0feee9574c88b6988fcaf607ab0c, user_id=9e5e965ac8ca4a1d93886d722ff0b2cb, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.655445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:19.689239 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.696813 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1396/2787] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:19 2022] PUT /identity/v3/projects/40dc0feee9574c88b6988fcaf607ab0c/users/9e5e965ac8ca4a1d93886d722ff0b2cb/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:19.713620 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d446-f945-4f60-a7d1-f55c28d5cc11 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.739547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:19.888702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.891119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.891119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.891119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.891828 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.898600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3b5c-e051-4004-be37-c9d79b01bbad admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:19.936397 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1392/2788] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:19.938933 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:19.959922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7946c-a3f4-4db7-ba01-88f2e36768bb admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:19.970442 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:19.972072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:19.972072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:19.972458 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] PATH_INFO: `/v3/projects/40dc0feee9574c88b6988fcaf607ab0c/users/9e5e965ac8ca4a1d93886d722ff0b2cb/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:19.975962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc0feee9574c88b6988fcaf607ab0c, user_id=9e5e965ac8ca4a1d93886d722ff0b2cb, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:19.981607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:20.000772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1397/2789] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:19 2022] PUT /identity/v3/projects/40dc0feee9574c88b6988fcaf607ab0c/users/9e5e965ac8ca4a1d93886d722ff0b2cb/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:20.022300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:20.022816 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:20.023311 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:20.079785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] MFA Rules not processed for user `9e5e965ac8ca4a1d93886d722ff0b2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:20.084033 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:20.181653 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:20.183666 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:20.196866 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1393/2790] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:20.219783 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:20.221897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:20.291240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1f418-730a-48ce-84e1-4f93bd723d54 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:20.298329 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:20.299777 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:20.300273 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:20.300809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:20.304072 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:20.331520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d66162-b5ad-46ad-ae12-6475565e8aaf None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:20.338121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:20.339337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:20.339834 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:20.340322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:20.432120 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=100 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:20.436636 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:20.439867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:20.452294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:20.454831 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:20.460845 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1398/2791] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:20.474714 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1394/2792] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:22.493083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:22.506443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09b29b-9a65-46f0-9a8b-2fc23eea6168 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:22.525201 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:22.540612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:22.540612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:22.540612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:22.643251 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:22.662810 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:22.664511 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:22.676923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1399/2793] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:23.763380 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:23.793070 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35795201-c807-4233-88af-87a05130b1f4 service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:23.797827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:23.799342 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:23.800145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:23.800424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:23.815000 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:23.818087 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2dc4cd-02ed-4ba2-9e44-e986e481ca45 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:23.842191 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1395/2794] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:42:28.543303 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:28.672661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35d06c-5e4b-453a-81ef-1f8c56677a43 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:28.681404 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:28.682432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:28.682931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:28.683342 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] PATH_INFO: `/v3/users/a6bed078c4fe4016a0ecfcee6b62cbbe` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:28.685311 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bed078c4fe4016a0ecfcee6b62cbbe)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:28.688093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:28.792926 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bed078c4fe4016a0ecfcee6b62cbbe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:28.808386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bed078c4fe4016a0ecfcee6b62cbbe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:28.822046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bed078c4fe4016a0ecfcee6b62cbbe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:28.822900 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bed078c4fe4016a0ecfcee6b62cbbe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:28.846749 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bed078c4fe4016a0ecfcee6b62cbbe'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:28.856451 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1400/2795] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:28 2022] DELETE /identity/v3/users/a6bed078c4fe4016a0ecfcee6b62cbbe => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:28.870242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:28.872010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:28.872010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:28.921633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:28.925453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:29.083406 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1396/2796] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:29.105433 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:29.107650 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:29.257678 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ef39c-4353-4cfd-b3a5-6e219d81bdcc admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:29.264525 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:29.265527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:29.265910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:29.266274 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:29.276851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:29.290742 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:29.294147 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:29.318365 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1401/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:29.811010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:29.813062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:29.836165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db425d-b587-4bda-9a2c-9e9122ba938f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:29.843890 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:29.845032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:29.845555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:29.846001 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] PATH_INFO: `/v3/projects/8d170930e8c04a52be39723909f9ef94` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:29.847966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d170930e8c04a52be39723909f9ef94)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:29.851919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:29.946524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d170930e8c04a52be39723909f9ef94', 'request_id': 'req-e0316b6b-497a-44a7-9139-1a75673f9533'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:29.966749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d170930e8c04a52be39723909f9ef94', 'request_id': 'req-e0316b6b-497a-44a7-9139-1a75673f9533'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:29.981722 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d170930e8c04a52be39723909f9ef94', 'request_id': 'req-e0316b6b-497a-44a7-9139-1a75673f9533'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.006889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] The token cache is being invalidate because project 8d170930e8c04a52be39723909f9ef94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:30.008451 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.012144 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.015490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1397/2798] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:29 2022] DELETE /identity/v3/projects/8d170930e8c04a52be39723909f9ef94 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:30.020837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:30.033507 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08144144-d1b0-41b5-b932-e2cc8bc11eb5 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:30.150093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:30.151146 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:30.151602 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:30.152252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] PATH_INFO: `/v3/users/517dbc84dafa4349b904c6a7bfd794bb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:30.154319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=517dbc84dafa4349b904c6a7bfd794bb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:30.157540 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:30.253178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517dbc84dafa4349b904c6a7bfd794bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.270768 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517dbc84dafa4349b904c6a7bfd794bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.281875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517dbc84dafa4349b904c6a7bfd794bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.282733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517dbc84dafa4349b904c6a7bfd794bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.297086 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517dbc84dafa4349b904c6a7bfd794bb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.305114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1402/2799] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:30 2022] DELETE /identity/v3/users/517dbc84dafa4349b904c6a7bfd794bb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:30.734969 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:30.737530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:30.763510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0316b6b-497a-44a7-9139-1a75673f9533 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:30.886060 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:30.895262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:30.896382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:30.897032 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:30.897942 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] PATH_INFO: `/v3/projects/9fc82c4a8a6e418d8b77c0da88f372ce` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:30.900216 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc82c4a8a6e418d8b77c0da88f372ce)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:30.903795 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:30.973066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc82c4a8a6e418d8b77c0da88f372ce', 'request_id': 'req-012b8938-e900-41a7-b251-117f9b2fe515'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.985737 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc82c4a8a6e418d8b77c0da88f372ce', 'request_id': 'req-012b8938-e900-41a7-b251-117f9b2fe515'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:30.996968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc82c4a8a6e418d8b77c0da88f372ce', 'request_id': 'req-012b8938-e900-41a7-b251-117f9b2fe515'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.017534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] The token cache is being invalidate because project 9fc82c4a8a6e418d8b77c0da88f372ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:31.018037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.019089 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.021933 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1398/2800] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:30 2022] DELETE /identity/v3/projects/9fc82c4a8a6e418d8b77c0da88f372ce => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:31.030354 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:31.033257 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:31.054250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b61d7d-56bc-4ad2-8950-e4883f0c9a7b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:31.149943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:31.150946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:31.151500 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:31.152081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] PATH_INFO: `/v3/users/9e5e965ac8ca4a1d93886d722ff0b2cb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:31.154310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5e965ac8ca4a1d93886d722ff0b2cb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:31.157606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:31.180317 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:31.235585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5e965ac8ca4a1d93886d722ff0b2cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.250064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5e965ac8ca4a1d93886d722ff0b2cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.265799 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5e965ac8ca4a1d93886d722ff0b2cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.266821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5e965ac8ca4a1d93886d722ff0b2cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.291441 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-a9002be3-23ee-458d-93cb-cd009b61d68c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5e965ac8ca4a1d93886d722ff0b2cb'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:31.312537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1403/2801] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:31 2022] DELETE /identity/v3/users/9e5e965ac8ca4a1d93886d722ff0b2cb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:31.750085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:31.752846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:31.770648 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b8938-e900-41a7-b251-117f9b2fe515 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:31.889608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:31.891649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:31.893142 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:31.894413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] PATH_INFO: `/v3/projects/40dc0feee9574c88b6988fcaf607ab0c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:31.901717 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40dc0feee9574c88b6988fcaf607ab0c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:31.912797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.006363 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40dc0feee9574c88b6988fcaf607ab0c', 'request_id': 'req-20eb31a1-8044-490d-a7ca-109f38b74128'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:32.023475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40dc0feee9574c88b6988fcaf607ab0c', 'request_id': 'req-20eb31a1-8044-490d-a7ca-109f38b74128'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:32.032547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40dc0feee9574c88b6988fcaf607ab0c', 'request_id': 'req-20eb31a1-8044-490d-a7ca-109f38b74128'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:32.055175 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] The token cache is being invalidate because project 40dc0feee9574c88b6988fcaf607ab0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:32.055678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:32.056877 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-20eb31a1-8044-490d-a7ca-109f38b74128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:32.059533 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1399/2802] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:31 2022] DELETE /identity/v3/projects/40dc0feee9574c88b6988fcaf607ab0c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:42:32.071232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.072080 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.072243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.104820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:32.106727 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:32.221851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1404/2803] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:32.233902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.234366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.234941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.273164 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:32.275066 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:32.289045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1400/2804] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:32.294223 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.306009 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecd6c8-ca9c-454a-a29f-2b004c973499 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.314669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.315399 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.315615 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.316013 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.317077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:32.317552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.320027 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.341238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1405/2805] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:42:32.347026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.369192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c8126-cc2f-4c72-88bd-52a5411c28dd None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.400446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.401740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.401740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.402352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.403055 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.405227 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.416945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:32.447680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1401/2806] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:32 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:32.452645 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.466143 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1a3ad-ce82-4b39-9754-647af95a549c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.563398 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.564347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.564807 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.565272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.566310 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.570028 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.602960 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1406/2807] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:42:32 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:42:32.609431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.623980 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b29e4-822f-47a8-85de-c8da875db87b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.629467 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.630068 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.630441 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.630711 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.631368 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.633292 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.646643 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1402/2808] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:32.652629 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.669142 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e77ec0-87ce-4cf3-b82d-a9f0506bf0a2 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.676826 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.677503 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.677868 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.677992 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.678652 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.680721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a85e9-f3a7-47de-968a-8515c657543b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.683533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:32.701230 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1407/2809] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:42:32.704152 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:32.721475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ca218-505e-476e-8e0c-b358d4e6f550 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:32.729442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:32.730495 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.730928 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.731400 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] PATH_INFO: `/v3/projects/dc8ba34da0124f3fb026ba0ab186dd26/users/6c1351fce0484d2bb3d73130c34b17fb/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.756748 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8ba34da0124f3fb026ba0ab186dd26, user_id=6c1351fce0484d2bb3d73130c34b17fb, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:32.760543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:32.783121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1403/2810] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:42:32 2022] PUT /identity/v3/projects/dc8ba34da0124f3fb026ba0ab186dd26/users/6c1351fce0484d2bb3d73130c34b17fb/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:32.800151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:32.800496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:32.800946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:32.847109 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] MFA Rules not processed for user `6c1351fce0484d2bb3d73130c34b17fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:32.849718 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:32.979682 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1408/2811] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:33.930911 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:33.933802 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:34.041990 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e89aa-c3f3-4e24-b19d-b72726a98a33 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:34.046862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:34.047558 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:34.047844 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:34.048121 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:34.059415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:34.060903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:34.071651 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1404/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:42.967416 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=33 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:44.066827 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:56.348496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:56.350314 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:56.422125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f714-f156-4b23-a627-aff48e9b899c None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:56.425942 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:56.426640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:56.426917 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:56.427187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:56.432062 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:56.526561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:56.528068 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930440f-5309-4fd9-9e81-3d25a0ddca2a service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:56.542312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1409/2813] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:58.130453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:58.131985 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:58.211548 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38806e-941e-4518-8fef-7230451d31cf service glance] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:58.218331 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:58.219915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:58.220361 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:58.221083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] PATH_INFO: `/v3/users/184d916641594820a3556ac4999c1060` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:58.222898 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=184d916641594820a3556ac4999c1060)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:58.225325 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:58.308785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184d916641594820a3556ac4999c1060'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:58.333945 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184d916641594820a3556ac4999c1060'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:58.349725 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184d916641594820a3556ac4999c1060'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:58.350995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184d916641594820a3556ac4999c1060'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:58.352500 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184d916641594820a3556ac4999c1060'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:58.377411 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1405/2814] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:42:58 2022] DELETE /identity/v3/users/184d916641594820a3556ac4999c1060 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:58.381815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:58.382394 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:58.382658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:58.415910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:58.417931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:58.526499 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1410/2815] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:58.552770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:58.555392 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:58.658383 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996885bd-43e8-4efa-b9ed-389261958f70 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:58.664968 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:58.666114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:58.666617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:58.667375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:58.681040 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:58.683751 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe408f3-e394-4d11-a143-0adb70e99563 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:58.696872 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1406/2816] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:59.428206 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:59.430661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:59.453831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038e2a-8fe7-4279-9cec-08337839bcb9 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:59.463427 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:59.464453 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:59.465060 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:59.465463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] PATH_INFO: `/v3/projects/e0aadf5390e64d64bf16c4cb283aadca` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:59.467655 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0aadf5390e64d64bf16c4cb283aadca)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:59.471051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:59.560443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:59.565302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0aadf5390e64d64bf16c4cb283aadca', 'request_id': 'req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:59.573904 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0aadf5390e64d64bf16c4cb283aadca', 'request_id': 'req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:59.589712 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0aadf5390e64d64bf16c4cb283aadca', 'request_id': 'req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:59.616672 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] The token cache is being invalidate because project e0aadf5390e64d64bf16c4cb283aadca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:42:59.617234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:59.618417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-62ca3c64-70d9-4a2c-8441-e1f6776d69dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:42:59.624696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1411/2817] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:42:59 2022] DELETE /identity/v3/projects/e0aadf5390e64d64bf16c4cb283aadca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:42:59.631581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:59.632536 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:59.632829 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:59.665585 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:59.667546 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:59.695893 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:59.784205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1407/2818] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:59.859987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:59.860333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:59.860987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:59.892133 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:59.894599 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:59.909253 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1412/2819] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:42:59.914731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:59.938833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6cf20-3dea-42e0-8087-a14889fb4ce5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:59.947812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:59.948915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:59.949328 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:59.950061 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:59.951248 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:42:59.951900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:59.954485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:59.971675 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1408/2820] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:42:59.978566 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.001181 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862a8cb-ee0c-410a-9fbf-cb41c2413be5 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.022759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.026534 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.027573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.028298 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.029610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.033471 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.116247 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1413/2821] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:42:59 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:00.124792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.143135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90400f5a-f3c4-4c9e-a5c4-adf48bc0c281 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.247901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.248510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.248820 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.249095 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.250521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.252759 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.284318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1409/2822] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:00 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:00.289258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.302328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557c669-9ba6-41e3-b61b-25e606b28b41 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.307715 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.308467 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.308801 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.309077 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.309767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.311520 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.324440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1414/2823] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:00.330278 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.348761 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25090-457c-4495-a3e7-c5909d4fc66d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.355532 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f3226-9611-447c-a599-896961a481de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.356195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3226-9611-447c-a599-896961a481de admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.356617 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3226-9611-447c-a599-896961a481de admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.357107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f3226-9611-447c-a599-896961a481de admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.357862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f3226-9611-447c-a599-896961a481de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.359487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f3226-9611-447c-a599-896961a481de admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.372756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1410/2824] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:00.377402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.390299 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8988f-8f23-4075-9ef6-1e9419f98e52 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.396571 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.397336 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.397616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.397918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] PATH_INFO: `/v3/projects/6259b279d5f7431db80680bdd68c6567/users/79ec67a084d44ac2ab1df0ee1440aac8/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.415267 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6259b279d5f7431db80680bdd68c6567, user_id=79ec67a084d44ac2ab1df0ee1440aac8, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.417975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.433984 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1415/2825] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:00 2022] PUT /identity/v3/projects/6259b279d5f7431db80680bdd68c6567/users/79ec67a084d44ac2ab1df0ee1440aac8/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:00.449957 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.450319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.450632 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.485624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] MFA Rules not processed for user `79ec67a084d44ac2ab1df0ee1440aac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:00.487114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:00.583919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1411/2826] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:00.589869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.676902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c9dd-46e6-408d-a688-bf614fc68eed admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.682767 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.683396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.683676 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.683972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.684705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.686658 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.717935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1416/2827] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:43:00 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:00.723217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.736770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d6cf-e701-4616-b932-94705db01e8f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.751200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:00.815493 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.816113 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.816393 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.816698 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.817693 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.821012 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.855062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1412/2828] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:00 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:00.859465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.875982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1a6f1-4288-408a-acca-5af15a2356d8 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.883595 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.885135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.885488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.885821 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.886721 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.888611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.902798 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1417/2829] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:43:00.912962 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.925345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04978b3e-54d2-42c5-b98c-0e06807ca30b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.933289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.934352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.934843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.935333 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:00.936274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:00.939085 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6883c-b6e5-43db-884e-1c86b44cfbe3 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:00.955584 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1413/2830] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:00.961543 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:00.979647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d592b-dcfc-4b9e-ad61-0861eb60a283 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:00.986422 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:00.987178 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:00.987526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:00.987872 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] PATH_INFO: `/v3/projects/2e7cb70f683c473b8f42684f6e74240a/users/04279445e24c4503bc6d3e8db484e979/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.009594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7cb70f683c473b8f42684f6e74240a, user_id=04279445e24c4503bc6d3e8db484e979, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.013973 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.033614 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1418/2831] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:00 2022] PUT /identity/v3/projects/2e7cb70f683c473b8f42684f6e74240a/users/04279445e24c4503bc6d3e8db484e979/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:01.045786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.046072 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.046345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.081649 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] MFA Rules not processed for user `04279445e24c4503bc6d3e8db484e979`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:01.083029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:01.183105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1414/2832] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:01.190516 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.220564 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:01.292241 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509289-b798-4ce2-ab23-42e46122278a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.297987 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.298607 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.298885 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.299151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.299788 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.301901 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.348363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1419/2833] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:43:01 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:01.356289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.371710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb7ecc-0a39-44f5-b92d-f5f1123b4703 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.469045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.469978 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.470412 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.470862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.472224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.479345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.504611 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1415/2834] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:01 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:43:01.511656 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.527046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098a029-7239-4a06-bdac-4e7506f5024c admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.538204 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.539197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.539637 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.540090 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.541603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.545163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.565898 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1420/2835] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:01.576971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.590104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff3082-3916-4cfd-841d-8ae6baac3a2f admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.595534 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.596266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.596712 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.597096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] PATH_INFO: `/v3/projects/b51d9d7861ea41f2a0ef6c8030592186/users/f81c3f23818c4d538cef9f597b267958/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.615958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:01.624906 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51d9d7861ea41f2a0ef6c8030592186, user_id=f81c3f23818c4d538cef9f597b267958, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.635988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.668476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1416/2836] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:01 2022] PUT /identity/v3/projects/b51d9d7861ea41f2a0ef6c8030592186/users/f81c3f23818c4d538cef9f597b267958/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:01.672549 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.690370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aea67-ef23-4c86-859c-09f7662cafe7 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.785812 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.787052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.787052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.787052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.787703 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.789460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6b3a9-f870-494f-9b2f-21d31b53f916 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.806931 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1421/2837] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:43:01.811605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:01.824622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17273bfb-bf46-4611-80b2-d00c1af9840b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:01.832682 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:01.833622 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.834042 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.834565 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] PATH_INFO: `/v3/projects/b51d9d7861ea41f2a0ef6c8030592186/users/f81c3f23818c4d538cef9f597b267958/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.837352 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51d9d7861ea41f2a0ef6c8030592186, user_id=f81c3f23818c4d538cef9f597b267958, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:01.841875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:01.859680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1417/2838] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:01 2022] PUT /identity/v3/projects/b51d9d7861ea41f2a0ef6c8030592186/users/f81c3f23818c4d538cef9f597b267958/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:01.870679 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:01.871074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:01.871321 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:01.915593 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] MFA Rules not processed for user `f81c3f23818c4d538cef9f597b267958`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:01.917198 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:02.020232 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1422/2839] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:02.035560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:02.037940 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:02.150970 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac0c72-4da5-4e70-a38b-c3e70d651ec9 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:02.158938 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:02.159572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:02.159856 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:02.160131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:02.237424 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:02.239453 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:02.251551 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1418/2840] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:03.893994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:03.895276 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:03.913661 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733809-ff3b-4481-9072-2b0c6d9830d1 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:03.919513 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:03.920527 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:03.921046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:03.921545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:03.943506 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:04.006977 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:04.008671 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:04.023243 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1423/2841] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:05.401366 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:05.412071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5622130-ae0a-47b0-aefa-1e79f10de09a service cinder] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:05.418775 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:05.419884 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:05.420450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:05.421111 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:05.435282 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:05.443003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d399c3-a972-4b5e-a752-cc45add02ff6 service cinder] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:05.456812 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1419/2842] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:09.533972 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:09.606306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e71a7-4dd9-48e8-b717-c787fe2af813 service cinder] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:09.611789 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:09.612439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:09.612971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:09.613419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] PATH_INFO: `/v3/users/79ec67a084d44ac2ab1df0ee1440aac8` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:09.614733 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ec67a084d44ac2ab1df0ee1440aac8)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:09.616705 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:09.683976 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ec67a084d44ac2ab1df0ee1440aac8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:09.692634 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ec67a084d44ac2ab1df0ee1440aac8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:09.709410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ec67a084d44ac2ab1df0ee1440aac8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:09.710496 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ec67a084d44ac2ab1df0ee1440aac8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:09.730893 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ec67a084d44ac2ab1df0ee1440aac8'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:09.740493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1424/2843] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:43:09 2022] DELETE /identity/v3/users/79ec67a084d44ac2ab1df0ee1440aac8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:09.745971 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:09.746510 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:09.747037 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:09.782885 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:09.784252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:09.857646 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:09.872800 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1420/2844] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:09.883852 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:09.886159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:09.931515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:09.974538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6826a-ca77-441a-a727-5a4e088c5b13 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:09.978497 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:09.979114 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:09.979396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:09.979668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:09.990396 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:09.991871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:10.003425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1425/2845] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:10.321271 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:10.322345 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:10.338294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405e851-9b73-45cb-9c3c-950f18cdab0f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:10.344941 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:10.346008 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:10.346492 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:10.346855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] PATH_INFO: `/v3/projects/6259b279d5f7431db80680bdd68c6567` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:10.349114 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6259b279d5f7431db80680bdd68c6567)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:10.353071 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:10.413544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6259b279d5f7431db80680bdd68c6567', 'request_id': 'req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.423102 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6259b279d5f7431db80680bdd68c6567', 'request_id': 'req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.429527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6259b279d5f7431db80680bdd68c6567', 'request_id': 'req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.444554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] The token cache is being invalidate because project 6259b279d5f7431db80680bdd68c6567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:43:10.445034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.447304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.447304 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1421/2846] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:43:10 2022] DELETE /identity/v3/projects/6259b279d5f7431db80680bdd68c6567 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:10.455156 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:10.466410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ade962-1913-44f8-a9c7-16b22f5ba1a0 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:10.538452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:10.539250 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:10.539392 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:10.540159 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] PATH_INFO: `/v3/users/04279445e24c4503bc6d3e8db484e979` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:10.541448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04279445e24c4503bc6d3e8db484e979)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:10.543585 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:10.603083 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04279445e24c4503bc6d3e8db484e979'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.617125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04279445e24c4503bc6d3e8db484e979'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.625693 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04279445e24c4503bc6d3e8db484e979'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.626425 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04279445e24c4503bc6d3e8db484e979'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.640954 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04279445e24c4503bc6d3e8db484e979'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:10.649067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1426/2847] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:43:10 2022] DELETE /identity/v3/users/04279445e24c4503bc6d3e8db484e979 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:11.041322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:11.043883 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:11.065958 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96139b8-80d5-48fe-ba80-02fe38f1b0f8 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:11.158260 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:11.159335 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:11.159545 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:11.159830 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] PATH_INFO: `/v3/projects/2e7cb70f683c473b8f42684f6e74240a` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:11.161103 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7cb70f683c473b8f42684f6e74240a)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:11.163057 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:11.233860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7cb70f683c473b8f42684f6e74240a', 'request_id': 'req-a863bdd2-787c-4a82-b312-ad84baea4716'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.244744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7cb70f683c473b8f42684f6e74240a', 'request_id': 'req-a863bdd2-787c-4a82-b312-ad84baea4716'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.253069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7cb70f683c473b8f42684f6e74240a', 'request_id': 'req-a863bdd2-787c-4a82-b312-ad84baea4716'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.266234 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] The token cache is being invalidate because project 2e7cb70f683c473b8f42684f6e74240a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:43:11.266520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.267222 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.269062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1422/2848] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:43:11 2022] DELETE /identity/v3/projects/2e7cb70f683c473b8f42684f6e74240a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:11.273907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:11.277925 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:11.294067 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7a8dc-a597-4342-90b0-0bb8a60c7527 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:11.360328 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:11.370621 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:11.371349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:11.371888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:11.372164 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] PATH_INFO: `/v3/users/f81c3f23818c4d538cef9f597b267958` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:11.373370 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81c3f23818c4d538cef9f597b267958)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:11.375308 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:11.438837 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81c3f23818c4d538cef9f597b267958'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.448519 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81c3f23818c4d538cef9f597b267958'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.457581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81c3f23818c4d538cef9f597b267958'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.458493 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81c3f23818c4d538cef9f597b267958'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.477035 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-00c98b83-7858-4ff9-9f3d-677c4f6fcc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81c3f23818c4d538cef9f597b267958'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.485818 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1427/2849] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:43:11 2022] DELETE /identity/v3/users/f81c3f23818c4d538cef9f597b267958 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:11.806009 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:11.808604 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:11.827843 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863bdd2-787c-4a82-b312-ad84baea4716 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:11.840450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:11.918107 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:11.919130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:11.919475 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:11.919838 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] PATH_INFO: `/v3/projects/b51d9d7861ea41f2a0ef6c8030592186` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:11.922624 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51d9d7861ea41f2a0ef6c8030592186)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:11.926570 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:11.991685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51d9d7861ea41f2a0ef6c8030592186', 'request_id': 'req-e4713a7a-c227-4455-8f6d-04f54fd2a85d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:11.999821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51d9d7861ea41f2a0ef6c8030592186', 'request_id': 'req-e4713a7a-c227-4455-8f6d-04f54fd2a85d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.008506 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51d9d7861ea41f2a0ef6c8030592186', 'request_id': 'req-e4713a7a-c227-4455-8f6d-04f54fd2a85d'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.021799 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] The token cache is being invalidate because project b51d9d7861ea41f2a0ef6c8030592186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:43:12.022137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.022956 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.024920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1423/2850] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:43:11 2022] DELETE /identity/v3/projects/b51d9d7861ea41f2a0ef6c8030592186 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:12.030174 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb561cb9-a8a6-4202-b16d-933bea74567e None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.030684 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb561cb9-a8a6-4202-b16d-933bea74567e None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.030841 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb561cb9-a8a6-4202-b16d-933bea74567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.052780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-bb561cb9-a8a6-4202-b16d-933bea74567e None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:12.054193 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-bb561cb9-a8a6-4202-b16d-933bea74567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.096937 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.098903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.148449 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1428/2851] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:12.157411 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.157740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.157940 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.181238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:12.182459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.193576 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1429/2852] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:12.195308 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713a7a-c227-4455-8f6d-04f54fd2a85d admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.198196 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.204231 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.205365 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.205861 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.206347 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] PATH_INFO: `/v3/users/6c1351fce0484d2bb3d73130c34b17fb` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.208724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1351fce0484d2bb3d73130c34b17fb)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.209783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9c71d-4252-4aad-ac7a-42dfabc8aeac None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.212833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.215261 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.215993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.216272 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.216708 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.217445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81537) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:43:12.217753 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.220022 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.233122 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1430/2853] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:43:12.237795 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.248508 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727004c9-6bc3-45cc-8803-06dc0987ae9f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.253626 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.254578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.254578 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.255234 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.255440 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.257573 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.293836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:12.296110 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1431/2854] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:43:12.297341 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1351fce0484d2bb3d73130c34b17fb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.305414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.325606 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1351fce0484d2bb3d73130c34b17fb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.341803 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1351fce0484d2bb3d73130c34b17fb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.344577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1351fce0484d2bb3d73130c34b17fb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.346213 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-28b8a1c7-7735-457d-a376-23ac9ee95e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1351fce0484d2bb3d73130c34b17fb'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:12.351785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:12.365163 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1424/2855] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:43:12 2022] DELETE /identity/v3/users/6c1351fce0484d2bb3d73130c34b17fb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:12.372389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.372687 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.372979 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.397857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:12.399275 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.400831 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83294da-31a4-42ab-aaf3-d46bc853f26b admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.474959 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1425/2856] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:12.483150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.483811 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.484113 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.484640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] PATH_INFO: `/v3/users` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.485044 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.486792 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.486958 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.488262 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.522432 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1432/2857] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:43:12.524304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.526271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.545907 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd16cb-eed5-4d80-86d4-f97d3c47c005 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.552834 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.553754 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.554043 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.554473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.555282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.557610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.561915 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd317e6-b76b-4136-bcb9-3636ecfcb427 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.568982 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.569920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.570892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.570892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.580716 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1433/2858] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:43:12.583390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.589634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.591313 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.601772 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ee34d-5af3-4671-bc0b-8abbcc8ef0dd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.603303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1426/2859] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:43:12.610344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.611364 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.611843 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.612616 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.613584 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.615265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d5b4a-c8e1-4b0b-be05-7faed7c7a470 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.632625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1434/2860] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:43:12.640195 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.651948 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b671531-e0f5-4521-a7ae-42c99929d2eb service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.658288 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.658919 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.659201 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.659710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] PATH_INFO: `/v3/projects/84b949156c47430fae4f516e4c6d80ba/users/ec929ddc024a4790b7091f60e4733cd4/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.679355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b949156c47430fae4f516e4c6d80ba, user_id=ec929ddc024a4790b7091f60e4733cd4, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.682154 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.699355 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1427/2861] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:12 2022] PUT /identity/v3/projects/84b949156c47430fae4f516e4c6d80ba/users/ec929ddc024a4790b7091f60e4733cd4/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:12.704809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.705138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.705395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.727344 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:12.728792 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:12.829859 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1435/2862] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:12.840524 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.908266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:12.924034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057147e-6ae6-40e2-bbdc-cd2a4ebdb69b admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:12.930137 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:12.931295 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:12.931862 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:12.933318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:12.949322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:12.951863 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66c289-9361-476c-9e41-4e581e208b68 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:12.970626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1428/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:12.990158 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:12.992759 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:13.014462 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1f59-ab12-419c-83f4-8ea515c210b3 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.026809 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.028000 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.028532 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.033971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] PATH_INFO: `/v3/projects/dc8ba34da0124f3fb026ba0ab186dd26` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.033971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8ba34da0124f3fb026ba0ab186dd26)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.036545 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.102827 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:13.112245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8ba34da0124f3fb026ba0ab186dd26', 'request_id': 'req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:13.122860 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8ba34da0124f3fb026ba0ab186dd26', 'request_id': 'req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:13.136491 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8ba34da0124f3fb026ba0ab186dd26', 'request_id': 'req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:13.154579 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] The token cache is being invalidate because project dc8ba34da0124f3fb026ba0ab186dd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:43:13.155215 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:13.156989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-4acd2bc5-6a06-40d1-8188-3b5787e45bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:13.160632 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1436/2864] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:43:12 2022] DELETE /identity/v3/projects/dc8ba34da0124f3fb026ba0ab186dd26 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:13.169949 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.170858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.170858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.200481 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:13.203735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:13.329389 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1429/2865] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:13.332226 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.332646 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.332873 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.358971 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:13.360923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:13.370046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1437/2866] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:13.376166 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.393450 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19454fed-2f7b-4ec2-856c-21e03af2c68d None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.400569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.401623 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.402097 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.402577 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.403684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:43:13.404189 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.406750 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.419936 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1430/2867] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:43:13.426302 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.443385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28140-c9cb-4062-800b-ca68aa545d06 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.449014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.449975 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.450528 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.450743 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.451408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.454032 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.496956 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1438/2868] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:43:13 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:13.502240 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.516724 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41de6-c293-4e0c-a6b7-871e0d6af42c admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.590005 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.590629 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.590908 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.591180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.591833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.593852 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.628738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1431/2869] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:13 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:13.633713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.651390 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ad3d-8017-4f11-a250-3350fc3b7b2d admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.657461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.658315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.658551 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.659450 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.659793 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.661929 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.675167 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1439/2870] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:43:13.679319 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.694145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee14dd-c071-46a3-b958-8e0630529040 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.701819 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.702608 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.703357 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.703735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.704414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.706853 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb3c27-3c4a-46fc-b24f-af0b70c180c9 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.724456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1432/2871] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:13.731802 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:13.751330 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54b1fa-56fe-428a-9dd6-156c904d6104 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:13.759388 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:13.760064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.760346 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.760654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] PATH_INFO: `/v3/projects/04128f08e2cc45c0b1ccd87598cb9684/users/f58cb5a4d3e34ae5a1d046c34c36c8e1/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.782125 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] RBAC: Authorizing `identity:create_grant(project_id=04128f08e2cc45c0b1ccd87598cb9684, user_id=f58cb5a4d3e34ae5a1d046c34c36c8e1, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:13.784439 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:13.806553 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1440/2872] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:13 2022] PUT /identity/v3/projects/04128f08e2cc45c0b1ccd87598cb9684/users/f58cb5a4d3e34ae5a1d046c34c36c8e1/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:13.823138 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:13.823421 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:13.823692 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:13.861150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] MFA Rules not processed for user `f58cb5a4d3e34ae5a1d046c34c36c8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:13.862645 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:13.952864 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:14.001157 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1433/2873] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:14.023294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:14.027004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:14.132902 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a4efc-5d3b-428b-9a70-f2842fe835be admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:14.139937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:14.139937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:14.139937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:14.139937 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:14.150006 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:14.151577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317e169-5996-4381-b422-c3104291fca8 service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:14.172221 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1441/2874] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:14.212092 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:14.316256 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914aa6a-bbf7-47cb-a100-66aa94e54fe5 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:14.322778 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:14.323781 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:14.324247 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:14.324770 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] PATH_INFO: `/v3/users/f58cb5a4d3e34ae5a1d046c34c36c8e1` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:14.326780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58cb5a4d3e34ae5a1d046c34c36c8e1)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:14.329640 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:14.444274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58cb5a4d3e34ae5a1d046c34c36c8e1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:14.464265 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58cb5a4d3e34ae5a1d046c34c36c8e1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:14.472710 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58cb5a4d3e34ae5a1d046c34c36c8e1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:14.473516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58cb5a4d3e34ae5a1d046c34c36c8e1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:14.474145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58cb5a4d3e34ae5a1d046c34c36c8e1'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:14.499056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1434/2875] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:43:14 2022] DELETE /identity/v3/users/f58cb5a4d3e34ae5a1d046c34c36c8e1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:43:14.506065 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:14.506385 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:14.506778 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:14.542554 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:14.545229 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:14.650134 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1442/2876] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:14.665544 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:14.667203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:14.754156 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57966dff-fd15-4b05-8f7b-8b1f2093bcc4 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:14.762135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:14.762868 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:14.763206 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:14.763490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:14.780285 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:14.784780 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e099c7-ec4c-488a-a708-757dcffa6226 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:14.807454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1435/2877] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:43:15.318526 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:15.320969 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:15.354881 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69544d7-f538-4d85-9cce-3ab5d2ec4069 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:15.374548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:15.377959 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:15.378644 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:15.379187 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] PATH_INFO: `/v3/projects/04128f08e2cc45c0b1ccd87598cb9684` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:15.381381 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04128f08e2cc45c0b1ccd87598cb9684)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:15.385932 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:15.401306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:15.482979 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04128f08e2cc45c0b1ccd87598cb9684', 'request_id': 'req-9346125c-c1d0-436a-9fd9-106cedfcbe95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:15.495371 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04128f08e2cc45c0b1ccd87598cb9684', 'request_id': 'req-9346125c-c1d0-436a-9fd9-106cedfcbe95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:15.505148 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04128f08e2cc45c0b1ccd87598cb9684', 'request_id': 'req-9346125c-c1d0-436a-9fd9-106cedfcbe95'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:15.528011 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] The token cache is being invalidate because project 04128f08e2cc45c0b1ccd87598cb9684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:43:15.528363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:15.529046 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:43:15.531548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1443/2878] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:43:15 2022] DELETE /identity/v3/projects/04128f08e2cc45c0b1ccd87598cb9684 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:18.758119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:18.759276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:18.759276 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:18.769559 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:18.802603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] MFA Rules not processed for user `ec929ddc024a4790b7091f60e4733cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:18.803846 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:18.896738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1436/2879] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:18.902056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:18.903504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:18.995236 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346125c-c1d0-436a-9fd9-106cedfcbe95 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.000490 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.001152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.001952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.001952 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.002343 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.004319 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.038780 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1444/2880] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:19.043523 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:19.056552 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4b447-b16f-494f-ac88-72b492c57a3f None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.139653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.141636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.141636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.141636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.142075 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.146119 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.172376 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1437/2881] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:43:19 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:43:19.177111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:19.192435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73ef8d-fd65-4865-92ec-ec114af14636 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.199157 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.199785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.200075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.200332 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.201787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.204265 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.223183 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1445/2882] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:19.228464 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:19.240633 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77bc120-3c37-45d5-9399-c675685e27ee admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.246627 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.247273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.247555 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.247984 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] PATH_INFO: `/v3/projects/a8a79363ee1f4ea49e63d4010b777cea/users/54dad68cdf3d4e1a927029faa9cfc973/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.267205 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a79363ee1f4ea49e63d4010b777cea, user_id=54dad68cdf3d4e1a927029faa9cfc973, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.270516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.287540 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1438/2883] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:19 2022] PUT /identity/v3/projects/a8a79363ee1f4ea49e63d4010b777cea/users/54dad68cdf3d4e1a927029faa9cfc973/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:19.293957 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:19.304208 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f581-4d4e-4127-924c-f9b21d9ec927 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.387214 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.387857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.388145 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.388419 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.389138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.391731 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68324c9a-33dc-419f-958b-b4c7be4c6aec admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.404659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1446/2884] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:43:19.409897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:19.427932 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149a78-7c09-4afe-babd-ebdfc5d49e14 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:19.440452 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:19.441503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:19.441503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:19.445168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] PATH_INFO: `/v3/projects/a8a79363ee1f4ea49e63d4010b777cea/users/54dad68cdf3d4e1a927029faa9cfc973/roles/1036ab5f43fc4e2ba01419fa66bbfb11` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:19.445168 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a79363ee1f4ea49e63d4010b777cea, user_id=54dad68cdf3d4e1a927029faa9cfc973, role_id=1036ab5f43fc4e2ba01419fa66bbfb11)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:19.446833 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:19.462521 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1439/2885] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:43:19 2022] PUT /identity/v3/projects/a8a79363ee1f4ea49e63d4010b777cea/users/54dad68cdf3d4e1a927029faa9cfc973/roles/1036ab5f43fc4e2ba01419fa66bbfb11 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:43:26.138463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7733-1dcf-45af-b130-82564f60a9d1 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:26.139056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7733-1dcf-45af-b130-82564f60a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:26.139056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7733-1dcf-45af-b130-82564f60a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:26.148996 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:26.176051 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d26a7733-1dcf-45af-b130-82564f60a9d1 None None] MFA Rules not processed for user `54dad68cdf3d4e1a927029faa9cfc973`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:26.177403 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d26a7733-1dcf-45af-b130-82564f60a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:26.280014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1447/2886] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:26.293756 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:26.295581 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:26.369488 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:26.394349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760f218-de06-49a0-8eda-d59382501ff7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:26.398601 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abda3580f52421d816eac7bee95b492', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:26.399237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:26.399519 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:26.399821 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:26.489228 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:26.491024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:26.504842 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1440/2887] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:43:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:43:26.709312 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037e264-5d7c-4d01-99b1-317500ace649 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:26.711681 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037e264-5d7c-4d01-99b1-317500ace649 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:26.713938 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037e264-5d7c-4d01-99b1-317500ace649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:26.771726 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-8037e264-5d7c-4d01-99b1-317500ace649 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:26.773918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8037e264-5d7c-4d01-99b1-317500ace649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:26.902459 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1448/2888] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:43:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:26.969076 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:26.970520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:27.063529 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfcab5-d8a4-454f-bcdb-44a404537981 service glance] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:27.069677 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:27.070459 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:27.070731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:27.071011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:27.082105 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:27.085203 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557a4d3-273e-4a27-95b3-53ba5d406507 service swift] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:27.099069 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1441/2889] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:43:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:43:36.880163 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=61 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:37.100563 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=60 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:37.639366 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628500-3130-4b0a-8677-af905cb24182 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:37.639722 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628500-3130-4b0a-8677-af905cb24182 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:37.639998 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52628500-3130-4b0a-8677-af905cb24182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:37.661817 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-52628500-3130-4b0a-8677-af905cb24182 None None] MFA Rules not processed for user `ec929ddc024a4790b7091f60e4733cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:37.663069 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-52628500-3130-4b0a-8677-af905cb24182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:37.675518 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1449/2890] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:43:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:43:40.694835 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:40.698182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:40.698182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:40.733997 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:43:40.736180 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:43:40.750554 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1442/2891] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 29 21:43:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:43:40.779630 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52628500-3130-4b0a-8677-af905cb24182 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:43:40.803552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52628500-3130-4b0a-8677-af905cb24182 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:43:40.813081 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:43:40.814130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:43:40.814640 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:43:40.815129 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:43:40.831237 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:43:40.834429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:43:40.863804 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1450/2892] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:43:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:43:50.724442 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:43:50.846281 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=9 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:03.602736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:03.604274 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:03.693847 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17ef4b-cd1d-448e-9290-c61e56ddecf1 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:03.699056 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:03.699684 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:03.699964 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:03.700237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:03.703954 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:03.789011 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:03.790583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00540385-34fb-4c44-b1ee-31f6b9a8de17 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:03.803851 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1443/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:05.055747 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:05.072218 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e298a-e09f-4835-829c-d48d704beb31 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:05.078252 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:05.079445 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:05.079903 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:05.080334 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:05.083339 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:05.102120 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:05.103594 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:05.117647 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1451/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:05.970509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:05.971375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:05.971375 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:05.997268 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] MFA Rules not processed for user `eb53ae825eea452d9d456233ac2af43c`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:44:05.999280 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:06.087024 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1444/2895] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:44:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:44:15.111836 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:16.077242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=79 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:16.493092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:16.505562 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b0589-3099-4190-9fe4-5d10ae0a4c84 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:16.509891 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:16.510613 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:16.510964 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:16.511386 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:16.525989 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:16.527815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:16.546890 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1452/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:26.540874 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=6 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:35.598542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:35.609634 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fda78d-c9e7-4ece-8796-0ef267c639d2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:35.613461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:35.614064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:35.614336 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:35.614603 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:35.616560 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:35.628730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:35.630388 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:35.639605 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1445/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:45.635364 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=6 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:50.978460 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:50.988245 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426027b5-ae71-4a9d-98ff-a965cdcab6d3 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:50.992101 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:50.992776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:50.993059 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:50.993333 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:50.995100 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:51.068955 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:51.070393 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:51.088205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1453/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:58.602875 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:58.604422 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:58.619924 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba0d3-8162-4730-b110-b439829d160a service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:58.626104 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:58.626744 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:58.627028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:58.627303 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] PATH_INFO: `/v3/users/d882c87d5de7497b822ec7b1798d2008` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:58.628456 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d882c87d5de7497b822ec7b1798d2008)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:58.630549 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:58.679485 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd882c87d5de7497b822ec7b1798d2008'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:58.692403 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd882c87d5de7497b822ec7b1798d2008'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:58.699457 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd882c87d5de7497b822ec7b1798d2008'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:58.700200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd882c87d5de7497b822ec7b1798d2008'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:58.700857 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd882c87d5de7497b822ec7b1798d2008'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:58.714140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1446/2899] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:44:58 2022] DELETE /identity/v3/users/d882c87d5de7497b822ec7b1798d2008 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:44:58.899347 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:58.911410 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43799-82d4-43b7-96c7-af22d9109c85 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:58.975948 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:58.976701 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:58.977055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:58.977402 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] PATH_INFO: `/v3/projects/98d4703c6737454291724a5ad15c9cbb` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:58.978597 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d4703c6737454291724a5ad15c9cbb)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:58.980620 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:59.035295 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d4703c6737454291724a5ad15c9cbb', 'request_id': 'req-45da8aac-f669-416d-968d-4235727f7a5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.040888 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d4703c6737454291724a5ad15c9cbb', 'request_id': 'req-45da8aac-f669-416d-968d-4235727f7a5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.050641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d4703c6737454291724a5ad15c9cbb', 'request_id': 'req-45da8aac-f669-416d-968d-4235727f7a5f'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.063765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] The token cache is being invalidate because project 98d4703c6737454291724a5ad15c9cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:44:59.064135 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.065552 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.066949 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1454/2900] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:44:58 2022] DELETE /identity/v3/projects/98d4703c6737454291724a5ad15c9cbb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:44:59.071064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:59.072825 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:59.097527 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd500ce7-2ae5-412b-befd-82cc041b5c17 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:59.173150 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:59.174135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:59.174636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:59.175266 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] PATH_INFO: `/v3/users/4100e961720a41d7b46ea0cd5b5f9a45` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:59.176887 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=4100e961720a41d7b46ea0cd5b5f9a45)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:59.180007 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:59.248626 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4100e961720a41d7b46ea0cd5b5f9a45'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.267145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4100e961720a41d7b46ea0cd5b5f9a45'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.281238 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4100e961720a41d7b46ea0cd5b5f9a45'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.283445 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4100e961720a41d7b46ea0cd5b5f9a45'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.284512 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4100e961720a41d7b46ea0cd5b5f9a45'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.286636 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:59.326882 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1447/2901] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:44:59 2022] DELETE /identity/v3/users/4100e961720a41d7b46ea0cd5b5f9a45 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:44:59.544152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:59.546986 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:59.560570 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da8aac-f669-416d-968d-4235727f7a5f admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:59.584246 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:59.625641 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:59.626070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:59.626389 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:59.626823 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] PATH_INFO: `/v3/projects/2c25a88a2c3541c7a2928901758bda71` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:59.628225 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c25a88a2c3541c7a2928901758bda71)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:59.630239 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:59.681322 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c25a88a2c3541c7a2928901758bda71', 'request_id': 'req-ffcfc004-d1f7-4833-87bc-29261818a77a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.686822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c25a88a2c3541c7a2928901758bda71', 'request_id': 'req-ffcfc004-d1f7-4833-87bc-29261818a77a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.695463 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c25a88a2c3541c7a2928901758bda71', 'request_id': 'req-ffcfc004-d1f7-4833-87bc-29261818a77a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.707750 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] The token cache is being invalidate because project 2c25a88a2c3541c7a2928901758bda71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:44:59.708173 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.708875 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:44:59.710414 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1455/2902] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:44:59 2022] DELETE /identity/v3/projects/2c25a88a2c3541c7a2928901758bda71 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:45:09.314413 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:09.363719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:09.363719 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:09.376749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abd9d-d1c9-41ee-9a71-64975049b0ac admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:09.455530 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:09.456578 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:09.457096 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:09.457583 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] PATH_INFO: `/v3/users/eb53ae825eea452d9d456233ac2af43c` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:09.459454 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb53ae825eea452d9d456233ac2af43c)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:09.463086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:09.532270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb53ae825eea452d9d456233ac2af43c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.545232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb53ae825eea452d9d456233ac2af43c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.553433 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb53ae825eea452d9d456233ac2af43c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.554165 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb53ae825eea452d9d456233ac2af43c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.554763 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-cf4c77c0-e6de-4bdd-9c37-a8bc89699de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb53ae825eea452d9d456233ac2af43c'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.569054 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1448/2903] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:45:09 2022] DELETE /identity/v3/users/eb53ae825eea452d9d456233ac2af43c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:45:09.704625 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:09.762443 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:09.763915 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:09.780064 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfc004-d1f7-4833-87bc-29261818a77a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:09.840918 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:09.841504 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:09.841785 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:09.842055 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] PATH_INFO: `/v3/projects/d05e27bd70e64afbbc58379e64826217` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:09.843191 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05e27bd70e64afbbc58379e64826217)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:09.845313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:09.898408 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05e27bd70e64afbbc58379e64826217', 'request_id': 'req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.904291 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05e27bd70e64afbbc58379e64826217', 'request_id': 'req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.915606 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05e27bd70e64afbbc58379e64826217', 'request_id': 'req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.929323 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] The token cache is being invalidate because project d05e27bd70e64afbbc58379e64826217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:45:09.929610 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.930280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-f7ffe7b4-51b9-41ff-b0b7-beefcef9c0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:09.931776 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1456/2904] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:45:09 2022] DELETE /identity/v3/projects/d05e27bd70e64afbbc58379e64826217 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:45:09.937736 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:09.938028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:09.938322 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:09.958858 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:45:09.960062 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:10.032414 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1449/2905] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:45:10.041612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.041922 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.042197 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.063828 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:45:10.065076 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:10.075819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1457/2906] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:45:10.083184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.094291 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821043e8-c7cf-423f-8622-07e05e3e2b98 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.100224 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.100865 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.101145 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.101415 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.102080 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81536) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:45:10.102378 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.104026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.117995 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1450/2907] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 29 21:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:45:10.122994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.133074 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067005-5cd9-438f-8343-0c760ac9d864 None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.138316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.138946 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.139222 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.139488 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] PATH_INFO: `/v3/projects` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.140130 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.142088 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.188249 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1458/2908] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 29 21:45:10 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:45:10.189569 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.199836 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8af05-bb7d-4a2e-9fc9-62d3d0a237a0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.203571 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:10.281272 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.281892 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.282170 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.282446 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.283098 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.285077 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.307518 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1451/2909] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 29 21:45:10 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:45:10.312034 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.322581 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814def8-333c-4653-bdb4-201a7d5f041e admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.327200 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.327799 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.328075 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.328349 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.329138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.330740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.341869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1459/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:45:10.346117 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.356786 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a57df-6d34-41d5-b35e-89a80cb251d7 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.362849 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.363600 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.363873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.364143 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.364841 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.370628 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24153b50-31bf-4da4-8e80-e548bb212025 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.382474 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1452/2911] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 29 21:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:45:10.387435 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.398557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36c85-9320-43d0-9550-bac839455e36 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.403271 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.404561 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.404871 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.405150 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] PATH_INFO: `/v3/projects/78ea9223adc64880a8040e2c842abf30/users/d328c1d39e0d4b6aa212b80eda5c65dd/roles/89fad9d5c83b4b04939f1ffc830b118d` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.421121 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ea9223adc64880a8040e2c842abf30, user_id=d328c1d39e0d4b6aa212b80eda5c65dd, role_id=89fad9d5c83b4b04939f1ffc830b118d)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.423783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.437657 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1460/2912] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 29 21:45:10 2022] PUT /identity/v3/projects/78ea9223adc64880a8040e2c842abf30/users/d328c1d39e0d4b6aa212b80eda5c65dd/roles/89fad9d5c83b4b04939f1ffc830b118d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:45:10.448174 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.448432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.448745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.479294 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] MFA Rules not processed for user `d328c1d39e0d4b6aa212b80eda5c65dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:45:10.480535 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:10.546929 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1453/2913] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:45:10.558085 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:10.559829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:10.580797 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:10.623966 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66595a7-bef8-4ece-adc6-b257c7cb6573 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:10.628103 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f82e19c26b45d2a4ac373665f8c9d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:10.628760 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:10.629038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:10.629315 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:10.639484 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:10.641093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8069a9d-3867-42a5-bf20-a20ed6f31bcf service cinder] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:10.650982 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1461/2914] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:45:20.544041 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:20.646654 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=18 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:27.160217 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:27.240064 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bd6fc-dbe3-4fbd-b62d-64fdb7b265d8 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:27.244537 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:27.245190 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:27.245466 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:27.245735 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] PATH_INFO: `/v3/users/d328c1d39e0d4b6aa212b80eda5c65dd` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:27.246901 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d328c1d39e0d4b6aa212b80eda5c65dd)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:27.248876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:27.313132 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd328c1d39e0d4b6aa212b80eda5c65dd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:27.326917 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd328c1d39e0d4b6aa212b80eda5c65dd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:27.334289 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd328c1d39e0d4b6aa212b80eda5c65dd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:27.335026 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd328c1d39e0d4b6aa212b80eda5c65dd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:27.335618 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd328c1d39e0d4b6aa212b80eda5c65dd'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:27.350184 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1454/2915] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:45:27 2022] DELETE /identity/v3/users/d328c1d39e0d4b6aa212b80eda5c65dd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:45:27.354740 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] REQUEST_METHOD: `POST` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:27.355313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:27.355313 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:27.374765 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.auth.core [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] MFA Rules not processed for user `3cc351f3cc8d45098e84aaabaeda36e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81537) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:45:27.376096 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:27.445015 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1462/2916] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 29 21:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:45:27.456130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:27.457655 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:27.524418 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15288f92-8c62-472d-aa9d-284cd7a83ecc admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:27.528344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:27.528989 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:27.529270 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:27.529547 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:27.540112 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:27.541599 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:27.551708 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1455/2917] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:45:27.922583 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:27.924131 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:27.937865 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3121c-8141-4f9a-aa69-dea9c95665be None None] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:27.947962 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:27.948699 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:27.949019 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:27.949935 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] PATH_INFO: `/v3/projects/78ea9223adc64880a8040e2c842abf30` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:27.951372 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ea9223adc64880a8040e2c842abf30)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:27.955202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:28.012555 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ea9223adc64880a8040e2c842abf30', 'request_id': 'req-99aa9802-22d7-4551-874f-96908764ca47'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:28.019094 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ea9223adc64880a8040e2c842abf30', 'request_id': 'req-99aa9802-22d7-4551-874f-96908764ca47'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:28.027688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ea9223adc64880a8040e2c842abf30', 'request_id': 'req-99aa9802-22d7-4551-874f-96908764ca47'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:28.047160 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] The token cache is being invalidate because project 78ea9223adc64880a8040e2c842abf30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81537) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:45:28.047622 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:28.048550 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:45:28.051355 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1463/2918] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:45:27 2022] DELETE /identity/v3/projects/78ea9223adc64880a8040e2c842abf30 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:45:31.759200 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:31.770283 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84787804-1486-4fd0-a4b2-c99538d682f0 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:31.818903 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:31.875732 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:31.876351 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:31.876706 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:31.876981 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:31.878837 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:31.894793 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:31.896680 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:31.907346 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1456/2919] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:45:32.020928 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:32.023052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:32.041482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa9802-22d7-4551-874f-96908764ca47 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:32.047038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:32.047994 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:32.048417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:32.048877 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:32.051557 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:32.152014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:32.153688 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:32.173869 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1464/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:45:32.265135 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:32.267381 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:32.354476 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eaec-2aec-4dd4-9340-2312f040cc62 service neutron] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:32.362029 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:32.363036 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:32.363487 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:32.364172 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:32.367242 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:32.383461 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:32.385572 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:32.398621 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1457/2921] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:45:36.051851 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:36.054205 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:36.078943 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4ae2d-01dc-4aad-8728-640616e68178 service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:36.088769 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:36.089659 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:36.090095 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:36.090687 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:36.093280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:36.111668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:36.118286 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:36.121437 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:36.151363 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1465/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:45:42.392920 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=48 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:45.501798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:45.533745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef801a-ed0a-4e8e-81e9-bf4cf3013897 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:45.549798 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:45.552516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:45.553516 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:45.555100 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:45.563431 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:45.596192 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:45.600063 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec10b9-ea64-4319-8506-166dbebcd1d6 service nova] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:45.631465 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1458/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:45:46.142665 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=2 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:45:47.170152 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:47.173429 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:47.192309 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2337-5783-4617-bf8f-d0385e4f24d7 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:47.199045 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0446b5664ae0453a8ccd57dfc562eb9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:47.200258 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:47.203482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:47.203482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:47.203482 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:47.222316 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:47.225195 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:47.245099 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1466/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:45:53.982508 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] REQUEST_METHOD: `POST` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:53.983106 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:53.983777 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:54.049850 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.auth.core [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] MFA Rules not processed for user `0af35a01c7c6427681a1f12508c1ff35`. Rule list: `[]` (Enabled: `True`). {{(pid=81536) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:45:54.051140 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:54.149975 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1459/2925] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 29 21:45:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:45:54.208038 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:54.209713 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:54.343004 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872c474-2092-49e0-9efa-78c53fa6db18 service nova] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:54.349461 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06c36d7a8a2246a38fe16e32a4607aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:54.350093 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:54.350374 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:54.350819 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:54.362612 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:54.364696 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:54.375863 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1467/2926] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 29 21:45:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:45:59.037003 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:45:59.038721 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:59.062873 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5155c-c80f-461c-a7dc-ff0262b7adc2 None None] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:45:59.071131 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:45:59.072130 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:45:59.072542 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:45:59.072950 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:45:59.075587 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:45:59.095520 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:45:59.098128 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:45:59.120745 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1460/2927] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:45:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:46:03.231839 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:03.234397 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:03.367387 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1625a8-9bf7-4615-b1a3-73c41061c787 service swift] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:03.372832 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:03.373544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:03.374202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:03.374202 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] PATH_INFO: `/v3/users/ec929ddc024a4790b7091f60e4733cd4` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:03.375558 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec929ddc024a4790b7091f60e4733cd4)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:03.377850 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:03.464306 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec929ddc024a4790b7091f60e4733cd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.477820 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec929ddc024a4790b7091f60e4733cd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.490704 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:03.496254 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec929ddc024a4790b7091f60e4733cd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.497238 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec929ddc024a4790b7091f60e4733cd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.513282 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec929ddc024a4790b7091f60e4733cd4'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.522517 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1468/2928] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:46:03 2022] DELETE /identity/v3/users/ec929ddc024a4790b7091f60e4733cd4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:03.736398 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:03.750730 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5108-3758-4f23-a675-52754e171918 service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:03.826740 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:03.827509 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:03.827966 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:03.828350 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] PATH_INFO: `/v3/projects/84b949156c47430fae4f516e4c6d80ba` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:03.829653 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b949156c47430fae4f516e4c6d80ba)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:03.831689 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:03.886312 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:03.888027 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b949156c47430fae4f516e4c6d80ba', 'request_id': 'req-be603e04-057c-42c6-973a-78f4f5bf7374'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.898254 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b949156c47430fae4f516e4c6d80ba', 'request_id': 'req-be603e04-057c-42c6-973a-78f4f5bf7374'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.907078 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b949156c47430fae4f516e4c6d80ba', 'request_id': 'req-be603e04-057c-42c6-973a-78f4f5bf7374'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.920010 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] The token cache is being invalidate because project 84b949156c47430fae4f516e4c6d80ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:46:03.920349 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.920986 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:03.922503 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1461/2929] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:46:03 2022] DELETE /identity/v3/projects/84b949156c47430fae4f516e4c6d80ba => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:03.926829 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:03.928466 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:03.943538 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c29c6-ed5b-4f43-9a35-b4f2a72939cd admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:04.016669 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:04.017304 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:04.017588 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:04.017857 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] PATH_INFO: `/v3/users/54dad68cdf3d4e1a927029faa9cfc973` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:04.019115 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] RBAC: Authorizing `identity:delete_user(user_id=54dad68cdf3d4e1a927029faa9cfc973)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:04.021335 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:04.084014 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54dad68cdf3d4e1a927029faa9cfc973'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.093269 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54dad68cdf3d4e1a927029faa9cfc973'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.101108 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54dad68cdf3d4e1a927029faa9cfc973'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.101867 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54dad68cdf3d4e1a927029faa9cfc973'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.116993 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54dad68cdf3d4e1a927029faa9cfc973'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.123603 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1469/2930] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:46:03 2022] DELETE /identity/v3/users/54dad68cdf3d4e1a927029faa9cfc973 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:04.361667 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:04.364394 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:04.390900 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be603e04-057c-42c6-973a-78f4f5bf7374 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:04.483685 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:04.484344 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:04.484652 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:04.484913 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] PATH_INFO: `/v3/projects/a8a79363ee1f4ea49e63d4010b777cea` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:04.486083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a79363ee1f4ea49e63d4010b777cea)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:04.488091 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:04.547749 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a79363ee1f4ea49e63d4010b777cea', 'request_id': 'req-7f611479-7d6b-4cee-8ecc-e154194d6022'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.557988 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a79363ee1f4ea49e63d4010b777cea', 'request_id': 'req-7f611479-7d6b-4cee-8ecc-e154194d6022'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.564902 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a79363ee1f4ea49e63d4010b777cea', 'request_id': 'req-7f611479-7d6b-4cee-8ecc-e154194d6022'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.580187 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] The token cache is being invalidate because project a8a79363ee1f4ea49e63d4010b777cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:46:04.582382 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.583232 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:04.585214 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1462/2931] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:46:04 2022] DELETE /identity/v3/projects/a8a79363ee1f4ea49e63d4010b777cea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:14.120495 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=62,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:14.574108 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:47:02.205111 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:02.207056 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:02.303787 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5d819-801e-434e-ac47-c5478fb656da admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:02.382240 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7af5dafacef4215b39c06af7c66c77c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:02.383537 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:02.384052 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:02.384533 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:02.388057 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:02.474766 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:02.477465 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:02.494951 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1470/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:47:10.648514 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:10.650607 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:10.737783 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:10.740522 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:10.768889 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1dc99-7191-46bf-85db-e80baecb685d service neutron] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:10.779469 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:10.780514 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:10.781066 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:10.781544 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] PATH_INFO: `/v3/users/4a20bdc6eee841258bd6ac11a8095a4a` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:10.783623 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a20bdc6eee841258bd6ac11a8095a4a)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:10.787138 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:10.794818 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611479-7d6b-4cee-8ecc-e154194d6022 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:10.802490 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:10.803300 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:10.803672 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:10.804045 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:10.806109 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:10.824125 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:10.826484 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:10.849225 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1463/2933] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:47:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:47:10.887280 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a20bdc6eee841258bd6ac11a8095a4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:10.902092 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a20bdc6eee841258bd6ac11a8095a4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:10.908668 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:47:10.910990 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a20bdc6eee841258bd6ac11a8095a4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:10.911782 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a20bdc6eee841258bd6ac11a8095a4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:10.933359 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a20bdc6eee841258bd6ac11a8095a4a'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:10.945452 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1471/2934] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:47:10 2022] DELETE /identity/v3/users/4a20bdc6eee841258bd6ac11a8095a4a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:47:11.185169 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:11.200318 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e239068-c117-40fd-8e8b-1a94e7aeb6ba service placement] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:11.294678 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:11.295337 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:11.295614 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:11.295889 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] PATH_INFO: `/v3/projects/2d9fa722c18045678015c7b300ccc3ab` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:11.297182 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9fa722c18045678015c7b300ccc3ab)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:11.299428 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:11.382701 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9fa722c18045678015c7b300ccc3ab', 'request_id': 'req-0609ea3d-ea6d-4998-837b-5c586c9f5276'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.394625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9fa722c18045678015c7b300ccc3ab', 'request_id': 'req-0609ea3d-ea6d-4998-837b-5c586c9f5276'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.403463 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9fa722c18045678015c7b300ccc3ab', 'request_id': 'req-0609ea3d-ea6d-4998-837b-5c586c9f5276'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.432432 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] The token cache is being invalidate because project 2d9fa722c18045678015c7b300ccc3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:47:11.432955 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.434273 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.435897 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1464/2935] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:47:11 2022] DELETE /identity/v3/projects/2d9fa722c18045678015c7b300ccc3ab => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:47:11.441473 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:11.443448 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:11.461126 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20273c-22ce-4b40-93fc-74037d87985a admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:11.542070 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:11.543268 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:11.543548 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:11.543822 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] PATH_INFO: `/v3/users/376759e72d214aeab86b4f605b913154` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:11.545331 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376759e72d214aeab86b4f605b913154)` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:11.547417 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:11.616611 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376759e72d214aeab86b4f605b913154'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.626151 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376759e72d214aeab86b4f605b913154'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.635049 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376759e72d214aeab86b4f605b913154'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.635830 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376759e72d214aeab86b4f605b913154'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.651395 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.notifications [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376759e72d214aeab86b4f605b913154'} {{(pid=81537) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:11.661154 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1472/2936] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 29 21:47:11 2022] DELETE /identity/v3/users/376759e72d214aeab86b4f605b913154 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:47:11.911683 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:11.913210 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:11.927252 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609ea3d-ea6d-4998-837b-5c586c9f5276 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:11.947755 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:47:12.002731 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cc351f3cc8d45098e84aaabaeda36e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '918734ce2c8c4362a4d1364fce951bb2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:12.003625 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:12.004086 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:12.004543 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] PATH_INFO: `/v3/projects/f5212b0ef07c4ad187500cffc36ff931` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:12.006438 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5212b0ef07c4ad187500cffc36ff931)` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:12.009785 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:12.076702 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5212b0ef07c4ad187500cffc36ff931', 'request_id': 'req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:12.087028 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5212b0ef07c4ad187500cffc36ff931', 'request_id': 'req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:12.095797 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5212b0ef07c4ad187500cffc36ff931', 'request_id': 'req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0'} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:12.110753 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] The token cache is being invalidate because project f5212b0ef07c4ad187500cffc36ff931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81536) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:47:12.111034 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:12.111876 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.notifications [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81536) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:47:12.113860 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1465/2937] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 29 21:47:11 2022] DELETE /identity/v3/projects/f5212b0ef07c4ad187500cffc36ff931 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:47:18.306923 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Authenticating user token {{(pid=81537) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:47:18.308805 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:18.380825 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:47:18.400515 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94325d-4c54-430e-90e2-53cf9cf38f64 admin admin] Validating token access rules against request {{(pid=81537) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:47:18.477815 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81537) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:47:18.478446 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] REQUEST_METHOD: `GET` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:47:18.478729 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] SCRIPT_NAME: `/identity` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:47:18.479007 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81537) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:47:18.481577 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.fernet_utils [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81537) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:47:18.494294 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:47:18.495910 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe17876-12bc-4aa2-965b-4a473204e88b service placement] RBAC: Authorization granted {{(pid=81537) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:47:18.506912 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: [pid: 81537|app: 0|req: 1473/2938] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:47:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:47:22.107237 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG dbcounter [-] [81536] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81536) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:47:28.501911 dv-cinder-854266-3-iscsi devstack@keystone.service[81537]: DEBUG dbcounter [-] [81537] Writing DB stats keystone:SELECT=32 {{(pid=81537) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:52:22.283329 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Authenticating user token {{(pid=81536) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:52:22.284808 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.fernet_utils [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81536) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:52:22.381659 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a10e5-c03e-47c1-ab0c-9bb22ce988e0 admin admin] Validating token access rules against request {{(pid=81536) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:52:22.386855 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0697bbbe44c74c538761737601ceeb10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daeb37b1a68a454294380a1715813f37', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81536) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:52:22.387458 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] REQUEST_METHOD: `GET` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:52:22.387738 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] SCRIPT_NAME: `/identity` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:52:22.388083 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81536) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:52:22.399385 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:52:22.401212 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea4dd-7cf3-4a2e-a5dd-a681a2f5dd8f service placement] RBAC: Authorization granted {{(pid=81536) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:52:22.414460 dv-cinder-854266-3-iscsi devstack@keystone.service[81536]: [pid: 81536|app: 0|req: 1466/2939] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 29 21:52:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)